-- Logs begin at Sun 2022-09-18 05:44:00 UTC, end at Sun 2022-09-18 07:12:32 UTC. -- Sep 18 06:09:57.325900 dv-cinder-855359-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 18 06:09:57.334439 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 06:09:57 2022] *** Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: nodename: dv-cinder-855359-3-iscsi Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: machine: x86_64 Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: clock source: unix Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: pcre jit disabled Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: detected number of CPU cores: 4 Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: current working directory: / Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: detected binary path: /usr/bin/uwsgi-core Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: your processes number limit is 47680 Sep 18 06:09:57.336398 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: your memory page size is 4096 bytes Sep 18 06:09:57.337010 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: detected max file descriptor number: 2048 Sep 18 06:09:57.337010 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: lock engine: pthread robust mutexes Sep 18 06:09:57.337010 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: thunder lock: enabled Sep 18 06:09:57.337010 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 18 06:09:57.337010 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 06:09:57.375505 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: Python main interpreter initialized at 0x55c425b499c0 Sep 18 06:09:57.375505 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: python threads support enabled Sep 18 06:09:57.375505 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: your server socket listen backlog is limited to 100 connections Sep 18 06:09:57.375505 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: your mercy for graceful operations on workers is 90 seconds Sep 18 06:09:57.375835 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 06:09:57.375927 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: *** Operational MODE: preforking *** Sep 18 06:09:57.376178 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: *** uWSGI is running in multiple interpreter mode *** Sep 18 06:09:57.376178 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: spawned uWSGI master process (pid: 81390) Sep 18 06:09:57.376364 dv-cinder-855359-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 18 06:09:57.382603 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: spawned uWSGI worker 1 (pid: 81397, cores: 1) Sep 18 06:09:57.382603 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: spawned uWSGI worker 2 (pid: 81398, cores: 1) Sep 18 06:09:57.382603 dv-cinder-855359-3-iscsi devstack@keystone.service[81390]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 06:10:01.628878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:10:01.628878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: from cryptography.utils import int_from_bytes Sep 18 06:10:01.629664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:10:01.629664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: from cryptography.utils import int_from_bytes Sep 18 06:10:01.634081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:10:01.634081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: from cryptography.utils import int_from_bytes Sep 18 06:10:01.634537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 06:10:01.634537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: from cryptography.utils import int_from_bytes Sep 18 06:10:01.678870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:10:01.679035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:10:01.679231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] command line args: [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:10:01.679496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:10:01.679672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ================================================================================ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:10:01.679922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] admin_token = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.680164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_dir = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.680348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.680526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_source = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.680702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] debug = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.680900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.682694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.682694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.682694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.682694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.682951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 06:10:01.683013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 06:10:01.683145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_config_append = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] command line args: [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 06:10:01.683265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_dir = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 06:10:01.683441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ================================================================================ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 06:10:01.683568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] admin_token = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.683891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_dir = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_source = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] debug = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.684983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_config_append = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.685981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_dir = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.686866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] publish_errors = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.687912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] standard_threads = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.688936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_journal = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_json = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_stderr = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.689900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_syslog = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.690089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.690312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.690404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.690501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] publish_errors = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.690589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.690743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.690905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.690986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.691165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.691250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.691370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.691454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.691602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.691680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.691833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] standard_threads = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.691916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.692066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.692144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.692307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.692383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.692507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.692587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.692648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_journal = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.692784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.692865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_json = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.692975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_stderr = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.693065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_syslog = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.693316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 06:10:01.693501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.693993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.694961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.695978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.696998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.697935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.698904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.699942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.700906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.701974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.702995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.703950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.704922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.705964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.706955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.707910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.708957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.709889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.710986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.711930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.712988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.713111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.713769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.713899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.714952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.715916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.external = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.716994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.password = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.token = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.717951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.718957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.719911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.external = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.720957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.password = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.token = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.721905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.722996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.723939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.724889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.725991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.726946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.727993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.728765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.729478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.729617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.729735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.729860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.729978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.730943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.731903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.732974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.733925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.734886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.735982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.736944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.737999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.738915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.739989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.740933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.741995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.742976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.743918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.744989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745821 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.745939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.746947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.747903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.748964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.749891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.750991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.751937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.752960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.753905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.754996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.755985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.756978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.757937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.758999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.759961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.760926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.761893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.762917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.763923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.764998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.765929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.766943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.767908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.768965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.769914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 06:10:01.769914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 06:10:01.794100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.794328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.794554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.795516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.795753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.795985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.802882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.804373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.830531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.830780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.832197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.832432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.884233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.884900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.933346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.933574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.934992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:01.935230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.089436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.089602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.089763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.089852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.089998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.090894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.113945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.114598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.125417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.125830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.138319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.138823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.139926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.140058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.140272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.140487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 18 06:10:02.156275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.156484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 06:10:02.156824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 06:10:02.157107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 06:10:02.157566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.157906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 18 06:10:02.158081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.158165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.158352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.158647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.158913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.168084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.168363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.168633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.168917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.169030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.169316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.169593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.169879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.175822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.176909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.178690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.178977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.179261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.179537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.179817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.179898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.180077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.180977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.181166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.181248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.181433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.181524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.181709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.182053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.182338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.182634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.204201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.205618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.205703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.205920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.207123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.207415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.210504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.210815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.211100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.212057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.212344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.212632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.217358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.217646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.219024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.219318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.223152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.223444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.223731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.224019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.224297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.224573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.224845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.224927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.225116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.225199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.225446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.225739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.226024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.226308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.226610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.226886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.240858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.241145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.242838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.243125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.244989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.247052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.248221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.248519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.250283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.250609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.252679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.254816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.255741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.258248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.258877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.259178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.259463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.259746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.260022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.260827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.262033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.262342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.262668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.262962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.263245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.263781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.276857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.277140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.277407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.277670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.279541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.279835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.280112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.280384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.284356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.285849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.287367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.287616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.287749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.289463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.290286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.290609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.290880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.291164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.291249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.292083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.292241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.295451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.295913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.296210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.296650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.297302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.303151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.303435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.303718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.304000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.304383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.304671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.304952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.310533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.310844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.311134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.311425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.311812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.312099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.312402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.320748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.323658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.326594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.327700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.328087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.329539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.329822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.330210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.330631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.333511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.334999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.336468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.336549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.336767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.337181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.337929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.340890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.341225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.341514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.341792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.343606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.345071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.347965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.348372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.348668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.348956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.349501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.349800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.350084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.357272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.357431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.357579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.357717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.357876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.358000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.358278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.358564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.358863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.359138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.359424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.359694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.359963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.360259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.360537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.360804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.361064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.364716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 18 06:10:02.365006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.365295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.365579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.365861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.366140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.366430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.366746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.367028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.367321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.367610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.367878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.368159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.368492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 18 06:10:02.395504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c425b499c0 pid: 81398 (default app) Sep 18 06:10:02.398795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9808-fa4d-4515-b2b9-68cab5a61484 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:02.399079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9808-fa4d-4515-b2b9-68cab5a61484 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:02.399345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d9808-fa4d-4515-b2b9-68cab5a61484 None None] PATH_INFO: `/v3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:02.404275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Sun Sep 18 06:10:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:02.406716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c425b499c0 pid: 81397 (default app) Sep 18 06:10:09.098774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:09.099291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:09.099291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:09.100611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 2/2] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:09.109767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:09.110362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:09.110362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:09.180503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: INFO dbcounter [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] Registered counter for database keystone Sep 18 06:10:09.199533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG oslo_db.sqlalchemy.engines [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:10:09.205544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writer thread running {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:10:09.386252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG passlib.handlers.bcrypt [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81397) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 18 06:10:09.386547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG passlib.handlers.bcrypt [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81397) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 18 06:10:09.443489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:09.445066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:09.570430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1/3] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 18 06:10:10.033820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:10.102718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: INFO dbcounter [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] Registered counter for database keystone Sep 18 06:10:10.124318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG oslo_db.sqlalchemy.engines [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81398) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 06:10:10.130490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writer thread running {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 06:10:10.252955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673145f-8c5a-4b44-9399-92dc2c1b39f9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:10.257247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:10.257869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:10.258142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:10.258408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:10.267156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:10.318820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 18 06:10:10.319105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a38a8-e5af-4ae6-9d4f-eb2726b9999c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:10.328359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 3/4] 10.222.0.20 () {58 vars in 1324 bytes} [Sun Sep 18 06:10:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:10:10.334287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:10.351255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f6a08-6414-495f-a329-8518f73b8e65 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:10.356373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:10.358244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:10.358678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:10.359090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] PATH_INFO: `/v3/projects/39ab4a771cc044ca930df75a3565efaa` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:10.361441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] RBAC: Authorizing `identity:get_project(project_id=39ab4a771cc044ca930df75a3565efaa)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:10.414715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 18 06:10:10.415057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b604447-6d13-4776-bde1-ac5e02caaf29 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:10.419609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 2/5] 10.222.0.20 () {56 vars in 1207 bytes} [Sun Sep 18 06:10:10 2022] GET /identity/v3/projects/39ab4a771cc044ca930df75a3565efaa => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:12.530249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b069a3-74fb-4627-b6f1-22aac0afad2b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:12.530832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b069a3-74fb-4627-b6f1-22aac0afad2b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:12.530832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b069a3-74fb-4627-b6f1-22aac0afad2b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:12.534889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 4/6] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:12.540162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:12.540534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:12.540714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:12.560692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:12.562192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:12.570034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 3/7] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 18 06:10:12.834096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b069a3-74fb-4627-b6f1-22aac0afad2b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:12.846528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b069a3-74fb-4627-b6f1-22aac0afad2b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:12.850702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:12.851290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:12.851566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:12.851834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:12.859334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:12.861084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914d58f-1e58-4bab-8fac-a6d33b69a59b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:12.869870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 5/8] 10.222.0.20 () {58 vars in 1324 bytes} [Sun Sep 18 06:10:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:10:12.875147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:12.884200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394081ab-17b0-461b-a642-ccabe74a0c09 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:12.889343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:12.889970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:12.890253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:12.890527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:12.891868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:12.893732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a737b1-c75c-4e8d-a4cf-cc175f8f6f4f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:12.895290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 4/9] 10.222.0.20 () {56 vars in 1198 bytes} [Sun Sep 18 06:10:12 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:15.096052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cada4-521a-4b36-bc1b-52951d9e7db4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.096591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cada4-521a-4b36-bc1b-52951d9e7db4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.096591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cada4-521a-4b36-bc1b-52951d9e7db4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.097452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94bd3-2acd-4f51-a4db-7cfee9445daf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.097822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94bd3-2acd-4f51-a4db-7cfee9445daf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.097999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b94bd3-2acd-4f51-a4db-7cfee9445daf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.098089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 6/10] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:15.099453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 5/11] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:15.104421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2897c3-1984-458c-8174-862fb073900b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.104705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2897c3-1984-458c-8174-862fb073900b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.104980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2897c3-1984-458c-8174-862fb073900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.105119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.105411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.105698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.123833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:15.125008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:15.146474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG passlib.handlers.bcrypt [None req-fc2897c3-1984-458c-8174-862fb073900b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81398) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 18 06:10:15.146904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG passlib.handlers.bcrypt [None req-fc2897c3-1984-458c-8174-862fb073900b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81398) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 18 06:10:15.196153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fc2897c3-1984-458c-8174-862fb073900b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:15.197783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fc2897c3-1984-458c-8174-862fb073900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:15.204348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:15.296689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:15.468130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.492841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:15.561820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2897c3-1984-458c-8174-862fb073900b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.570870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f5cb4-9e5e-4153-b7fe-2f81d729e37b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:15.605236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:15.605901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.606180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.606447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.617270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:15.626022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:15.627134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:15.627962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:15.632949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.662681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2897c3-1984-458c-8174-862fb073900b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:15.665033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:15.699322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:15.700173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.700450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.700717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.712502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:15.730707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd3cae-86a0-461e-bcf5-49e80a5617fb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:15.763542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:15.764396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.764673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.764940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.765609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:15.765906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:15.767275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:15.767377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:15.768787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:15.779280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:15.784863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.864852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 06:10:15.870534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.873241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:15.881966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2151c0-8101-4424-ade6-ff4227ba1e20 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:15.914719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:15.915457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:15.915732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:15.916003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:15.917661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:15.919137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:15.920674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:15.925894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:15.940873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:15.968014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801911b-7a7b-4daa-80ef-c9131115fc25 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:16.001842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:16.002642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:16.002921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:16.003199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:16.021099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:16.022553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:16.022700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc1ef4-3beb-44bf-8c72-cc2b7834b70e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:10:16.040522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:16.042015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:10:16.042879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:10:16.042879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:10:16.042879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:16.042879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:10:16.047993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:16.056606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:16.057331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:16.057614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:16.057898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:16.059042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:16.060479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:16.062035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:16.067573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:16.101965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:16.145445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:16.147046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9235a-2069-4689-82af-4b446f822075 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:16.163857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65dbac-5915-49f1-85ae-b9f8d4523bae None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:16.179999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:16.180601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:16.180881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:16.181153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:16.181792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:16.182634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:16.184030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e75274-c7a2-4297-9b2a-877f6cb26138 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:16.195451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:16.196052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:16.196330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:16.196598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:16.197311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.domain.id=default, user.id=949debb5f3604102a418027d122cd476) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:16.197660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:16.199615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3e026-c54f-4ce1-b9c4-1e8a18471bb4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:16.201603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:16.209867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Sun Sep 18 06:10:16 2022] GET /identity/v3/role_assignments?user.id=949debb5f3604102a418027d122cd476&scope.domain.id=default&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:18.536149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333227-c607-4692-8021-ae566fc16c70 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:18.537126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333227-c607-4692-8021-ae566fc16c70 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:18.537604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b333227-c607-4692-8021-ae566fc16c70 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:18.539191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:18.544714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d4cba-8e9e-4a6b-b5af-118c54babc95 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:18.545684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d4cba-8e9e-4a6b-b5af-118c54babc95 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:18.546155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d4cba-8e9e-4a6b-b5af-118c54babc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:18.569686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-387d4cba-8e9e-4a6b-b5af-118c54babc95 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:18.571229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-387d4cba-8e9e-4a6b-b5af-118c54babc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:18.647912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:18.827160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdaeac-7940-44b6-9cce-19afc7f0334c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:18.827826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdaeac-7940-44b6-9cce-19afc7f0334c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:18.828316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdaeac-7940-44b6-9cce-19afc7f0334c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:18.829868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:18.835134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94ff04-5226-4e4a-bcd5-f613bfb6a83d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:18.835662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94ff04-5226-4e4a-bcd5-f613bfb6a83d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:18.836150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94ff04-5226-4e4a-bcd5-f613bfb6a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:18.858270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7f94ff04-5226-4e4a-bcd5-f613bfb6a83d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:18.859788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f94ff04-5226-4e4a-bcd5-f613bfb6a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:18.947310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:19.015952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdaeac-7940-44b6-9cce-19afc7f0334c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.157154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdaeac-7940-44b6-9cce-19afc7f0334c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:19.182739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:19.201393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28d98-e1c2-4b92-b6de-11df513ba9ff None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.202044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28d98-e1c2-4b92-b6de-11df513ba9ff None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.202594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c28d98-e1c2-4b92-b6de-11df513ba9ff None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.204388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:19.208131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:19.208131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.208131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.208131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.221053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:19.227265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:19.228115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:19.228971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 13/27] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:19.231249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424b38-e12c-40d1-828c-e133a5f981f6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.231541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424b38-e12c-40d1-828c-e133a5f981f6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.231817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424b38-e12c-40d1-828c-e133a5f981f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.235323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.259862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b3424b38-e12c-40d1-828c-e133a5f981f6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:19.261693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b3424b38-e12c-40d1-828c-e133a5f981f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:19.271957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:19.365316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 15/28] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:19.370836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ea66-e492-44b8-bd5d-a4e054682d5a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.370836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ea66-e492-44b8-bd5d-a4e054682d5a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.370836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4ea66-e492-44b8-bd5d-a4e054682d5a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.370836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 16/29] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:19.376019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.376349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.376586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cac5b-639c-46a4-b789-ed16f3b49f4a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:19.376693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.422634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:19.424048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:19.440439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:19.441378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.441853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.442429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.443382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:19.443953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:19.445529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:19.472093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 14/30] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:19.475720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.542822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:19.564661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 17/31] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:19.565379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.671263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562e707-f95c-453e-93e7-a2f2dac99ad6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:19.709327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:19.732974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:19.733944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.734956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.735447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.747982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:19.749783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:10:19.767251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:19.768106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:10:19.768889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:10:19.768889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:10:19.768889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:19.770664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 15/32] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:19.779721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.800500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d2ac6-0dd1-4bf7-b3c1-c7628f3517df None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:19.886898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:19.899275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:19.899932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.900212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.900484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.901401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf05df-e73a-45bc-8c35-56c3bc2591d8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:19.902455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:19.904104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:19.906230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:19.907223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:19.944796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:19.945404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:19.945678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:19.946110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:19.946990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:19.947455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:19.950267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:19.965193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 16/34] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:19.966489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:20.031450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ebde3-4881-48d2-bca2-a1ca924d6c83 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:20.070365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:20.077476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:20.078415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.078863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.079277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.080090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:20.081762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:20.106488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2d435-c0be-4b13-a376-be90962c0870 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:20.141717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:20.142338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.142653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.142926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.144233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:20.145681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:20.147601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 17/35] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:20.148454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:20.168502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:20.224530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 19/36] 10.222.0.20 () {60 vars in 1131 bytes} [Sun Sep 18 06:10:19 2022] POST /identity/v3/roles => generated 219 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:20.225976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:20.253333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba557df-b4fe-477b-9cea-cb84f8509518 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:20.295928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:20.296577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.296855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.297178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.297869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:20.299541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbb0a9-ec39-48cd-9277-130577fb2488 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:20.391531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeccb64-a382-41e0-8e8d-9de47cd41e25 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:20.447455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:20.447455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.447455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.447455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.448575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:20.456037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:20.493510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 20/37] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:10:20 2022] POST /identity/v3/projects => generated 301 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:20.495264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:20.510600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 18/38] 10.222.0.20 () {60 vars in 1131 bytes} [Sun Sep 18 06:10:20 2022] POST /identity/v3/roles => generated 225 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:20.512328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adba7f-c224-47d8-8bca-e97325550194 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.512617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adba7f-c224-47d8-8bca-e97325550194 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.512894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adba7f-c224-47d8-8bca-e97325550194 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.514620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 19/39] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:20.520922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c023ac-d02b-44f6-986a-658e3aa47c03 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.521235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c023ac-d02b-44f6-986a-658e3aa47c03 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.521478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c023ac-d02b-44f6-986a-658e3aa47c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.555314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f9c023ac-d02b-44f6-986a-658e3aa47c03 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:20.557830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f9c023ac-d02b-44f6-986a-658e3aa47c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:20.675924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:20.682863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 20/40] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:20.737165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a4a3-053d-4b09-8982-fe6da73916e3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:20.760563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c4e6-b609-4e15-b6b5-f7a6d62ca2bd None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.760802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c4e6-b609-4e15-b6b5-f7a6d62ca2bd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.761184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980c4e6-b609-4e15-b6b5-f7a6d62ca2bd None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.762966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 21/41] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:20.771018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be680b1-7581-4736-836b-c53e95382739 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.772314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be680b1-7581-4736-836b-c53e95382739 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.772314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be680b1-7581-4736-836b-c53e95382739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.789617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:20.790332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:20.790644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:20.790928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] PATH_INFO: `/v3/domains/default/users/949debb5f3604102a418027d122cd476/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:20.793675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=949debb5f3604102a418027d122cd476, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:20.795174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4be680b1-7581-4736-836b-c53e95382739 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:20.795282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:20.796677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4be680b1-7581-4736-836b-c53e95382739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:20.799361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:20.820244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 21/42] 10.222.0.20 () {58 vars in 1361 bytes} [Sun Sep 18 06:10:20 2022] PUT /identity/v3/domains/default/users/949debb5f3604102a418027d122cd476/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:20.892159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 22/43] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:21.001269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:21.104596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f221dfc-2f27-473d-8553-c6859f476e9b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:21.106933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:21.141561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:21.142185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:21.142469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:21.142775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:21.143461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:21.144880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:21.159721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 22/44] 10.222.0.20 () {60 vars in 1131 bytes} [Sun Sep 18 06:10:21 2022] POST /identity/v3/roles => generated 223 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:21.165523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be680b1-7581-4736-836b-c53e95382739 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:21.273528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be680b1-7581-4736-836b-c53e95382739 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:21.317418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:21.318476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:21.318979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:21.319445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:21.320808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:21.322608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc154fb-a194-45c4-84eb-b6fc3d128c25 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:21.324428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 23/45] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:21.330009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:21.433861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98e61e-3ef2-4889-b96a-42b12eda60a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:21.467591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:21.468213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:21.468497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:21.468771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:21.469436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:21.470953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476855fc-a86e-42b5-814f-dacc0c1e2f21 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:21.496871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:10:21 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:10:22.945226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd533-06a0-4c59-b50f-eb35e5b04a03 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:22.945732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd533-06a0-4c59-b50f-eb35e5b04a03 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:22.945839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cd533-06a0-4c59-b50f-eb35e5b04a03 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:22.947137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:22.952555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:22.952963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:22.953113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:22.971598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:22.973117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:22.984123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:23.039462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:23.315373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd533-06a0-4c59-b50f-eb35e5b04a03 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:23.325551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:23.418182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cd533-06a0-4c59-b50f-eb35e5b04a03 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:23.450117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:23.450776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.451059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.451334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.464601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:23.469560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:23.470406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:23.471198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 25/49] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:23.476609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:23.577134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fac85-bffa-4f72-ac08-d9c771768eab None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:23.610395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:23.611045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.611334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.611610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.612287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:23.612591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:23.613977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa611b-b612-42e9-a0fa-56c3c92ce934 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:23.621992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 25/50] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:23.627712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:23.641166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790289-1469-4cdb-b270-34f2a14ad064 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.641453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790289-1469-4cdb-b270-34f2a14ad064 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.641729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff790289-1469-4cdb-b270-34f2a14ad064 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.643080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 26/51] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:23.647899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.648180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.648451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.666128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:23.667938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:23.669846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:23.695477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:23.730929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c340cdb-af06-48b5-acbc-bf5ccf7b7d31 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:23.735258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 27/52] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:23.762851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:23.763484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.763771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.764029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.765639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:23.767107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:23.768658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:23.774040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:23.869704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9956e-2499-46d3-b4c9-55a4f27eb50a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:23.902384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:23.903046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:23.903324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:23.903598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:23.904696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:23.906121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:23.907695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:23.913352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:24.000141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:24.009740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:24.011203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd0c6-b51f-47f9-b064-429c522b0878 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:24.044653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:24.045314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:24.045606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:24.045903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:24.046658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.domain.id=default, user.id=949debb5f3604102a418027d122cd476) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:24.046970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:24.048414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a80e7-76fb-4c87-a061-e502b34f70c5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:24.050463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:24.105299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642245a-48d3-4efd-9825-984a89fdaf2b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:24.142341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:24.142944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:24.143225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:24.143500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:24.144654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:24.146139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:24.147778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 29/55] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:24.153926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:24.162491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1322 bytes} [Sun Sep 18 06:10:23 2022] GET /identity/v3/role_assignments?user.id=949debb5f3604102a418027d122cd476&scope.domain.id=default&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 530 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:24.236858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:24.263915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd602a-2c83-4a15-80f2-e4a977afed5f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:24.298833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:24.299469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:24.299753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:24.300024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:24.300693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:24.302192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5513ce-b46f-4bd9-afae-69d380f8687d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:24.330870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 30/57] 10.222.0.20 () {60 vars in 1140 bytes} [Sun Sep 18 06:10:24 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:10:26.497330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e002eb-60d2-493e-a936-c15180348449 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:26.497772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e002eb-60d2-493e-a936-c15180348449 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:26.497877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e002eb-60d2-493e-a936-c15180348449 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:26.499250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:26.504346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:26.504714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:26.504908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:26.524023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:26.525221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:26.594531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:26.862942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e002eb-60d2-493e-a936-c15180348449 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:26.960048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e002eb-60d2-493e-a936-c15180348449 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:26.991903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:26.992525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:26.992789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:26.993061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:26.994164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:26.995628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542fc6a-be02-4aa5-a28b-fdd8f5aa9600 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:26.997149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:27.002338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:27.055620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:27.100409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afec3b-1886-4d6e-98ad-510a72da8b55 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:27.132200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:27.132746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:27.133030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:27.133306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:27.133967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:27.135986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fe183-1c4e-4921-b94e-dc86e930da9d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:27.161475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:27 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:30.345049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339b394-cd43-4ad6-a2dc-ab3e64ea4967 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.346048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339b394-cd43-4ad6-a2dc-ab3e64ea4967 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.346532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339b394-cd43-4ad6-a2dc-ab3e64ea4967 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.348710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:30.354288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fa299-1baf-4f8a-a756-4121ff1c5b52 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.354767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fa299-1baf-4f8a-a756-4121ff1c5b52 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.354920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41fa299-1baf-4f8a-a756-4121ff1c5b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.383781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d41fa299-1baf-4f8a-a756-4121ff1c5b52 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:30.385255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d41fa299-1baf-4f8a-a756-4121ff1c5b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:30.488578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee337e-5575-413c-b6b5-fe344d72fd93 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.489247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee337e-5575-413c-b6b5-fe344d72fd93 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.489737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee337e-5575-413c-b6b5-fe344d72fd93 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.490986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:30.491523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 31/64] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:30.498023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a4d2-6fa0-4a35-8fc5-de8a9c5ab50e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.498315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a4d2-6fa0-4a35-8fc5-de8a9c5ab50e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.498606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0a4d2-6fa0-4a35-8fc5-de8a9c5ab50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.528833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e7d0a4d2-6fa0-4a35-8fc5-de8a9c5ab50e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:30.530075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e7d0a4d2-6fa0-4a35-8fc5-de8a9c5ab50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:30.590731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:30.633265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:30.719038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d473-b901-4349-91b6-9116a8b20cb1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.719592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d473-b901-4349-91b6-9116a8b20cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.720031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d473-b901-4349-91b6-9116a8b20cb1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.721573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 32/66] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:30.726828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:30.727305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:30.727750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:30.747112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:30.748620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:30.775754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330d473-b901-4349-91b6-9116a8b20cb1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:30.855085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:30.891044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:30.950886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330d473-b901-4349-91b6-9116a8b20cb1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.000397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.001061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.001341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.001616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.009896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.016588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:31.021522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:31.022381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:31.023183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:10:31.029692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.138997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944080f4-9ebd-4e7b-a2ed-59f0430214e4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.165606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.173834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff37248-09ce-4f60-bf39-43f14c1376e0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.175430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.175774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.176048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.176331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.176604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.177272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:31.177577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.179009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6873af6-c482-4460-a1cc-cc9ec26a35b5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.188421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 34/69] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:31.189420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2370-0d79-48a2-a870-788e8049c907 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.189706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2370-0d79-48a2-a870-788e8049c907 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.189979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2370-0d79-48a2-a870-788e8049c907 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.195802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 35/70] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:31.199984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da2370-0d79-48a2-a870-788e8049c907 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.220789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.221423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.221705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.221973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.232711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ad5e30c6-2bc9-4dee-84c2-2f41292fdae6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:31.237527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:31.238550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:31.239764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 36/71] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:10:31.241103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81202eef-d33c-4cef-b3f0-d6da061952d2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.241349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81202eef-d33c-4cef-b3f0-d6da061952d2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.241618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81202eef-d33c-4cef-b3f0-d6da061952d2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.243031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 37/72] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:31.248082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.248365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.248640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.265791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:31.267014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:31.299950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da2370-0d79-48a2-a870-788e8049c907 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.332071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 38/73] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:31.332881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.334903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.335528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.335807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.336082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.337200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.338773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.340360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 36/74] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:31.341102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.404365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.441675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.452976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5750-c0ca-4604-a7fc-2b7c9ff65d96 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.456542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51780cbc-d9d2-440f-bc3a-82effc2c6874 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.488197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.488835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.489114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.489384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.495590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.496351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.496631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.497188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.497860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:31.498162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.499381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:31.499586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1f603171-9cf7-496f-8671-4fdcf6b7585b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:31.504155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:31.504965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:31.505706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:31.510645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 39/75] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:31.510843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 37/76] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:31.515920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.516032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.516032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.516032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.532590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:31.533785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:31.599572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 40/77] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:31.603550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.625616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8fff1-837f-4092-9aba-5b8cb5969907 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.643627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.660733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.661334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.661613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.661884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.691272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.711417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad41a4-69d6-417d-9a5a-6eb35e6bf740 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.736560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.738476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.746869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.747488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.747771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.748045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.748717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:31.749020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.750394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.759086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 41/78] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:31.764679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.848229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 38/79] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:31.849529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.864838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d442f0-1f98-4fe3-97ab-6030792ea883 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.868239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.898716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.899352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.899631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.899906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.913185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:31.914677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:31.916305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 42/80] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:31.917064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:31.919570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:31.950144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728061f0-265a-4ba0-8688-22fb687f3cba None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:31.982645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:31.983211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:31.983496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:31.983767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:31.993595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:31.999034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:31.999895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:32.000640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 39/81] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:32.001270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.013954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f76f3-b41a-4246-aab5-b35f8d88d296 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.046527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.047180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.047460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.047731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.048836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.050301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.051893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:31 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.055359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.077506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.099482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb0011-98c1-4698-9aa1-fabe8ba37b29 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.122012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.132596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.133291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.133544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.133751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.135504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.136945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.138590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 40/83] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.139338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.154920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743eaf24-dade-4dd6-8148-e072a0449902 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.187448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.188084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.188366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.188632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.189729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.191195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.192843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 44/84] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.194787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.242652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409861-7015-4fb4-b2ad-a6116df224d7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.276069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.276664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.276944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.277215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.290025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097763db-2bc7-43e0-b107-0f6683beaf47 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.292525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.323393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.324004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.324286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.324561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.325227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:32.325540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.326942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.331758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.338728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 45/85] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.339829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.347407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.348866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.435530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988aba8d-50e6-4cfb-ad3c-6fd374d9aedd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.444632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 41/86] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:32.445462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.475427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.476054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.476336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.476605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.492640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.540615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.543527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.545311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.547761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab0c24-1e0b-4671-a4f3-82a8f775500d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.581282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.581882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.582161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.582439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.583192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=949debb5f3604102a418027d122cd476) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:32.583510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.584972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.590756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 42/87] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/role_assignments?user.id=949debb5f3604102a418027d122cd476&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.591535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.674785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 46/88] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:32.675663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.707614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23778332-0f54-4817-9a72-d44df072ee20 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.719559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.746821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.747747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.748053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.748330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.749000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.750472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.781734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 43/89] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:32 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:32.781734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.811105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:32.816364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b213fb-ffe6-4bb6-a780-8e0196bfb6f9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.867130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.868098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.868626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.869051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.871207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.872911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.875110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 47/90] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.881037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.934831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407004d-cc6b-4169-bb64-b6c42c751c85 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:32.972046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:32.972643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:32.972922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:32.973193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:32.975114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:32.976691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:32.979171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 44/91] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:32.980408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:32.999527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f528f0-3e92-4a8d-bd01-91b0bd7e5715 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.018959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:33.044378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.045389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.045873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.046346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.047994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.049524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.051501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 48/92] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:33.052272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:33.102138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86579f-5d14-41ad-87c7-5aedf1b1d6dd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.103632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:33.150185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.150984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.151265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.151532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.184693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6930967-7bc0-4a1c-9814-bbc6b8605fc0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.224614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.225246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.225567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.225813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.226700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:33.227020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.228627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.234733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 49/93] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:33 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=d7498f66178947778670d4d051df8536 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:33.235551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:33.243687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.245714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.330974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:33.377547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:33.380807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46739026-a2b4-4953-968e-6af3b2fe6e8c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.429888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 45/94] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:33.446039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.447056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.448113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.448569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.449496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:33.452183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.453566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:33.467371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.481913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 50/95] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:33 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:33.585925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209811-7388-4394-8c58-b5e3f4722146 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.625893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.626865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.627321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.627823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] PATH_INFO: `/v3/projects/8c0ab0e0d28445ab8116d115510e894a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.629164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] RBAC: Authorizing `identity:get_project(project_id=8c0ab0e0d28445ab8116d115510e894a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.630858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759f433-7ee7-4934-8734-a94fd7b633d3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.633075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 46/96] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:33 2022] GET /identity/v3/projects/8c0ab0e0d28445ab8116d115510e894a => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:33.639318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:33.802481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc237-443f-4d52-ba62-e61db5e6ce02 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:33.845620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:33.846560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:33.847040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:33.847489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:33.848399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=8c0ab0e0d28445ab8116d115510e894a, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:33.848971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:33.854635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ff88a-a3ce-4c03-b4bb-915d31b586b5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:33.860811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 51/97] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:33 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=8c0ab0e0d28445ab8116d115510e894a&role.id=d7498f66178947778670d4d051df8536 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:35.805125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8eb8-eea3-4aca-b369-14f3958bc77e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:35.805125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8eb8-eea3-4aca-b369-14f3958bc77e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:35.805125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8eb8-eea3-4aca-b369-14f3958bc77e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:35.805125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 47/98] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:35.816003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b3e5-c67a-403d-9f02-f77168518a7b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:35.816832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b3e5-c67a-403d-9f02-f77168518a7b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:35.817320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b3e5-c67a-403d-9f02-f77168518a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:35.847846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bfa1b3e5-c67a-403d-9f02-f77168518a7b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:35.849370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bfa1b3e5-c67a-403d-9f02-f77168518a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:35.887325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:35.982970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 52/99] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:36.121724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0299dc-0808-4edd-b276-ef1edeee413f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.122385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0299dc-0808-4edd-b276-ef1edeee413f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.122899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0299dc-0808-4edd-b276-ef1edeee413f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.124439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 48/100] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:36.130014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc275-cfce-4c2f-a7eb-f08f91cc4469 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.130527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc275-cfce-4c2f-a7eb-f08f91cc4469 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.131057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dc275-cfce-4c2f-a7eb-f08f91cc4469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.151763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-596dc275-cfce-4c2f-a7eb-f08f91cc4469 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:36.152737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-596dc275-cfce-4c2f-a7eb-f08f91cc4469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:36.265092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 53/101] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:36.284217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e3ef-5e55-437e-a0c3-ecc200b8f14f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.284838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e3ef-5e55-437e-a0c3-ecc200b8f14f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.285311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e3ef-5e55-437e-a0c3-ecc200b8f14f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.286869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 49/102] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:36.292493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756382-5161-4f50-8021-07540971ea1e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.292946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756382-5161-4f50-8021-07540971ea1e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.293231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d756382-5161-4f50-8021-07540971ea1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.324061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7d756382-5161-4f50-8021-07540971ea1e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:36.325276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d756382-5161-4f50-8021-07540971ea1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:36.413200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 54/103] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:36.432397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258e3ef-5e55-437e-a0c3-ecc200b8f14f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:36.435618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:36.588687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78463b-29db-4fc7-a3d4-a3e7da4569ab None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.589009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78463b-29db-4fc7-a3d4-a3e7da4569ab None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.589253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78463b-29db-4fc7-a3d4-a3e7da4569ab None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.591042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 55/104] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:36.601287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.601287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.601287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.602366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258e3ef-5e55-437e-a0c3-ecc200b8f14f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:36.631562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:36.632769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:36.647448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:36.648465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.648942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.649408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.652945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:36.673024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:36.678985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:36.680068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:36.681672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:36.689895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 50/105] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:36.695561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:36.728017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 56/106] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:36.771650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:36.815457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:36.858671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78421e8e-8e56-42ae-9604-bbdaa1424a36 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:36.901823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:36.903377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.903809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.904233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.905048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38206e31-9501-48ec-b192-f55d27c8e9f0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:36.905487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:36.905936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:36.907750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:36.918412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 51/107] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:36.919608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:36.950556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:36.952228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.952517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.952795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.965889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9c655296-3680-4fcb-9e44-0c69452831a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:36.972709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:36.973530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:36.974290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:36.975967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 57/108] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:36.977489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcf1bc-da41-459c-b244-001c230067ea None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:36.977778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcf1bc-da41-459c-b244-001c230067ea None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:36.978056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcf1bc-da41-459c-b244-001c230067ea None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:36.979387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 58/109] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:36.983725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcf1bc-da41-459c-b244-001c230067ea None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:36.992185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.050943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209986a6-0763-4417-98ac-5b430ef1c207 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.076999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.093744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.094472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.095005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.095098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.096185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.097654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.098484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcf1bc-da41-459c-b244-001c230067ea None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.102756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 52/110] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.102756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.140847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.141492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.141779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.142480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.143662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.145812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8e418-3dcd-4252-beed-0e1d2c0f1b64 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.152346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 59/111] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:36 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.152346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.152346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.152346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.172722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:37.173922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:37.206931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d8160-9bbf-46a2-b70a-cc17a1a35634 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.249232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 60/112] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:37.249412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.249412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.249557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.249648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.249648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.249648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:37.249648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.249648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.252708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.262298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 53/113] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.263182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.334434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.366124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0d64a-2e80-4b56-ae78-48745fb16031 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.383768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3fc8d-57c3-4134-8804-100f3e781a9a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.406724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.407370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.407651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.407927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.419909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:37.421523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.422130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.422471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.422780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.423584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.425142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:37.425247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:37.426012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:37.426779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:37.430739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 61/114] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:37.432226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.465960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 54/115] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:10:37 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:37.475752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.547477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.571656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f45bdd-430c-42ad-87cc-798147b0c931 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.592336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f41fd3-cb3b-495f-b3ba-3d8009f1b548 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.601219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.609009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.609660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.609943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.610241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.611473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.613006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.618650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 62/116] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.618650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.640691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.641346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.641631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.641911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] PATH_INFO: `/v3/users/806b514454f44022942c7ebc4abc5cd4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.659046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=806b514454f44022942c7ebc4abc5cd4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.660534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51928b0-90cd-4255-84a7-eb2b2cb50ee7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.662454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 55/117] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/users/806b514454f44022942c7ebc4abc5cd4 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.663468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df45f9-3749-4ee7-afed-914d58a0c28d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.663825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df45f9-3749-4ee7-afed-914d58a0c28d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.664030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df45f9-3749-4ee7-afed-914d58a0c28d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.665245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 56/118] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:37.665945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df45f9-3749-4ee7-afed-914d58a0c28d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.732516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbde8d-4057-447d-9156-77e439b9ceae None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.767727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.768342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.768623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.768898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.769566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:37.769871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.771283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.772653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.780210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df45f9-3749-4ee7-afed-914d58a0c28d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.782473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 63/119] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:37.783514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.825427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.826497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.826497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.826846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.837344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:37.842722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:37.843589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:37.844372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 57/120] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:37.845004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:37.873583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:37.901190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dc5b5-fe6e-4bfd-bb6b-e15ad94fddf0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.938508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:37.939318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.939608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.939878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/949debb5f3604102a418027d122cd476/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.941972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, user_id=949debb5f3604102a418027d122cd476, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:37.943451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc8fdf-887a-4b65-8e55-122e25d39230 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:37.957307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 64/121] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:37 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/949debb5f3604102a418027d122cd476/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:37.957470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480b00b-890d-491a-9f08-75c96db0d48a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:37.968039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:37.968355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:37.968596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:37.997990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:37.999411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:38.007456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.008131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.008416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.008749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.086905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.087331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.090436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.097067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 65/122] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:38.097253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.147469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.238191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 58/123] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:38.239074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.249983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089243e-d1dd-49d8-9aaf-0599549decf5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.286609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.287257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.287545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.287822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.288984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.290450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.294314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 66/124] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.294970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.348306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84e35-c147-4f0f-af0c-ee823e102bcd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.351686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.375542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.394134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.395187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.395472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.395745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.396428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:38.396731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.398286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.423344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 59/125] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.423344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.436388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aea089-5d33-4744-bd2f-c05b43c2e768 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.493491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.494253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.494625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.494977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.496175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.497689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.499447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 67/126] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.500335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.558777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab085a-d029-4148-8fb3-70f14cc57373 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.599657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b38ef-2d33-4d10-a0fd-9bc20e60e6a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.603917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.604559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.604846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.605168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.606385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.608411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.610548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 60/127] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.611764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.628680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.643740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.644683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.644966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.645237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.647150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.648891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.651923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 68/128] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.652692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.679337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.745792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cc05-fe07-4d83-9d10-ed24d1705261 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.776832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f9143-3337-4e12-a08f-3e7f6571a381 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:38.790996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.791662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.792342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.792342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.805170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:38.811352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:38.812221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:38.813080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 61/129] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:38.813703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.818890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:38.820034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:38.820519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:38.821026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:38.822060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=806b514454f44022942c7ebc4abc5cd4) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:38.823177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:38.824897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:38.830996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 69/130] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/role_assignments?user.id=806b514454f44022942c7ebc4abc5cd4&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:38.832052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:38.945013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:38.993972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc322812-bce7-400e-b19d-57acdcafa521 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.010834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:39.042130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.042130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.042130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.042130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/3b6af059a40d42aa92232a0f849bc581/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.042490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, user_id=3b6af059a40d42aa92232a0f849bc581, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.043889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.052607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ef2d-6075-4549-9512-ebe8c76ab199 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.067174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 62/131] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:38 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/3b6af059a40d42aa92232a0f849bc581/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:39.068201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:39.128426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.129082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.129369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.129643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.130938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.132473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.134136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 70/132] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:38 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:39.152644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:39.271737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a6ff7-a90c-44d5-a254-dbfee5620db5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.298569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:39.311949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.312554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a5cc2-2643-4540-a3e3-76decbffbe04 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.312664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.312892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.313170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/3b6af059a40d42aa92232a0f849bc581/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.316837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, user_id=3b6af059a40d42aa92232a0f849bc581, role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.319277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.347252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 71/133] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:39 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/3b6af059a40d42aa92232a0f849bc581/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:10:39.392855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.393680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.393971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.394248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.394985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:39.395293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.396725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.411183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 63/134] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:39.416922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:39.586412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc87d7-2403-44eb-82fb-80e8f6e26ed1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.627529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.628173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.628447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.628715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.629852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.631392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.632977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 72/135] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:39 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:39.639242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:39.700585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:39.786267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e30d1-5ee5-42dc-9a03-1a7bdf41cbb0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:39.832450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:39.833085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:39.833368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:39.833642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] PATH_INFO: `/v3/projects/8c0ab0e0d28445ab8116d115510e894a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:39.834814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] RBAC: Authorizing `identity:get_project(project_id=8c0ab0e0d28445ab8116d115510e894a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:39.836271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42104cd8-b0ab-4234-afbf-138f06aa19f6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:39.838209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 64/136] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:39 2022] GET /identity/v3/projects/8c0ab0e0d28445ab8116d115510e894a => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:39.843585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:39.927751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:39.996762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d4190-91cd-45cf-9481-0a30d524ef7f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:40.019412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9d9-fa7f-4ea7-aed8-8d3cf4a8bd9f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.019901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9d9-fa7f-4ea7-aed8-8d3cf4a8bd9f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.019974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40c9d9-fa7f-4ea7-aed8-8d3cf4a8bd9f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.021437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 65/137] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:10:40.026872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.027154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.027428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.035572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:40.036314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.036593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.036865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] PATH_INFO: `/v3/projects/8c0ab0e0d28445ab8116d115510e894a/users/3b6af059a40d42aa92232a0f849bc581/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.038789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] RBAC: Authorizing `identity:create_grant(project_id=8c0ab0e0d28445ab8116d115510e894a, user_id=3b6af059a40d42aa92232a0f849bc581, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:40.040341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:40.061368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 73/138] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:39 2022] PUT /identity/v3/projects/8c0ab0e0d28445ab8116d115510e894a/users/3b6af059a40d42aa92232a0f849bc581/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:40.067768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:40.069021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:40.180158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 66/139] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:40.524027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:40.661196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c41da-8144-47e7-af06-0296b54104df None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:40.676095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:40.707254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:40.707889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.708166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.708429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.709542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:40.711009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be688-01ef-4bfa-92af-887e3abeb6ce None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:40.713080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 74/140] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:40.718696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:40.767821 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0d821-1df0-41b5-a742-50853014f5f4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.768110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0d821-1df0-41b5-a742-50853014f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.768385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0d821-1df0-41b5-a742-50853014f5f4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.769738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 75/141] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:40.776634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:40.777161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:40.777667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:40.779334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:40.800517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:40.802088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:40.916055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 76/142] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:40.951222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670801-af93-432c-a52a-704bd8d0d4d4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:41.005771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:41.006283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:41.006560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:41.006854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:41.007515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:41.008957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:41.040877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 67/143] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:40 2022] POST /identity/v3/users => generated 283 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:41.508657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:41.636286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195eb57e-e36c-457c-9956-23ae38de085d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:41.646713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:41.673769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:41.674404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:41.674709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:41.674980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:41.685729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:41.690499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:41.693724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:41.696784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 77/144] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:41.704124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:41.836316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:41.846144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf5ea-09cd-4e66-9d4a-4e8d64e5b2e1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:41.886872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:41.887525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:41.887810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:41.888078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:41.888732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:41.889031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:41.890681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:41.900737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 68/145] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:41.906559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:42.038199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff68beb-d696-41c0-bd61-229edfac35bf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:42.080047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:42.080964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:42.081406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:42.081867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] PATH_INFO: `/v3/users/949debb5f3604102a418027d122cd476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:42.083167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] RBAC: Authorizing `identity:get_user(user_id=949debb5f3604102a418027d122cd476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:42.084834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:42.086637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 78/146] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:41 2022] GET /identity/v3/users/949debb5f3604102a418027d122cd476 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:42.093621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:42.229252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6f909-c193-4360-836a-dc177793c6ec None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:42.269649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:42.270262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:42.270545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:42.270846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:42.298982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:42.355639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:42.357287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:42.477261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 69/147] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:42.483555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:42.522635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:42.599751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56108-fdff-408b-8f7f-c1d22a68af45 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:42.648457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:42.649109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:42.649393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:42.649671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:42.655100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:42.656576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f250c-0eae-46b5-a165-d8962d8805ec None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:42.658507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 79/148] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:42 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:42.664448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:42.729729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:42.787804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d9268-9aa9-4c05-9d1d-7053f462628b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:42.826270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:42.826908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:42.827192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:42.827465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:42.828199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=949debb5f3604102a418027d122cd476) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:42.828507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:42.829960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9d36-44af-4de6-b952-d1e254697200 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:42.836463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 70/149] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:42 2022] GET /identity/v3/role_assignments?user.id=949debb5f3604102a418027d122cd476&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:43.007203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf3081-7c6f-482a-9366-d0b8d3c08ed3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.007388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf3081-7c6f-482a-9366-d0b8d3c08ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.007669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf3081-7c6f-482a-9366-d0b8d3c08ed3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.009462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 80/150] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:43.016337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b978-c5cd-4fbf-a0e1-16f7bd36cc4e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.016619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b978-c5cd-4fbf-a0e1-16f7bd36cc4e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.016891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034b978-c5cd-4fbf-a0e1-16f7bd36cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.044482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d034b978-c5cd-4fbf-a0e1-16f7bd36cc4e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:43.045709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d034b978-c5cd-4fbf-a0e1-16f7bd36cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:43.134721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 71/151] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:43.144156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec98-84f0-4080-9c2c-18aa085f3813 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.144785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec98-84f0-4080-9c2c-18aa085f3813 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.145277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455ec98-84f0-4080-9c2c-18aa085f3813 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.149125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 81/152] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:43.152511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b6cd-4e40-4e1f-bcb8-aca7efe39573 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.152801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b6cd-4e40-4e1f-bcb8-aca7efe39573 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.153077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2b6cd-4e40-4e1f-bcb8-aca7efe39573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.175618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-19d2b6cd-4e40-4e1f-bcb8-aca7efe39573 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:43.176825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-19d2b6cd-4e40-4e1f-bcb8-aca7efe39573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:43.231686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:43.265232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 72/153] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:43.281962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d903d-742f-40e3-8bd0-dc419ace4034 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.282275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d903d-742f-40e3-8bd0-dc419ace4034 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.282506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d903d-742f-40e3-8bd0-dc419ace4034 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.283878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 82/154] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:43.300285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.300584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.300840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.331696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:43.332916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:43.417290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 73/155] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:43.563782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d903d-742f-40e3-8bd0-dc419ace4034 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:43.691219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d903d-742f-40e3-8bd0-dc419ace4034 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:43.694314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:43.730308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:43.731354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.731854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.732341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.743326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:43.748591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:43.749473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:43.750272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:43.752542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 83/156] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:43.759678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:43.854764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:43.880773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d5f51-2c84-43ea-8c2e-e2d3cce8a88e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:43.888621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:43.940211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:43.940833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:43.941112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:43.941384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:43.942066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:43.942367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:43.943781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:43.959980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 74/157] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:43.961019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.051718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89292efd-514e-4ac8-8d4e-c94e2cd4ecdc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.089366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.090392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.090932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.091367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.103403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e9a49fcc-e99c-4ab4-90e4-b1ae9553af3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:44.108985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:44.110054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:44.111057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:44.113287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 84/158] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:44.115010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffef4c-dff8-4d0e-bdcf-57393f06fa70 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.115459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffef4c-dff8-4d0e-bdcf-57393f06fa70 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.116244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffef4c-dff8-4d0e-bdcf-57393f06fa70 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.119268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 85/159] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:44.125025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffef4c-dff8-4d0e-bdcf-57393f06fa70 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.156270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:44.176565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4e15b-8882-4655-9f6a-cbb936e72b22 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.242994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffef4c-dff8-4d0e-bdcf-57393f06fa70 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.281487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.282704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.283131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.283545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.284992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:44.286749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:44.288631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 86/160] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:44 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:44.289646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.312747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.313417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.313694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.314021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.355201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2f762fbc-36b3-486a-bd5f-e0e7f08b94f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:44.360771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:44.361714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:44.362472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 75/161] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:44.363216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.363216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.363763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.409838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6741ba-97ba-4472-8eff-a2480f745ee2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.418774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:44.430111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:44.433002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:44.482882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.486613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.486978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.487323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.488076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:44.488450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:44.491149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:44.494998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:44.515114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 87/162] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:44.516134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.594411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 76/163] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:44.599483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.736273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5d6c8-c822-42db-9cd9-dfec97665acb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.776358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78385-6089-45d4-95d0-63222f26f2ca None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:44.812178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.813266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.813732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.814181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.814342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:44.814975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:44.818609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:44.818609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:44.818609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:44.818609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:44.818609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:44.828082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 77/164] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:44.828983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:44.858928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:44.870729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:44.909835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:44.911378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.013211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 88/165] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:45.018809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.074905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9011cb-c3ce-4de3-93b0-3a948019052d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.091185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.135819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.136464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.136735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.137006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] PATH_INFO: `/v3/users/806b514454f44022942c7ebc4abc5cd4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.138152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] RBAC: Authorizing `identity:get_user(user_id=806b514454f44022942c7ebc4abc5cd4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.139873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.141627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 78/166] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:44 2022] GET /identity/v3/users/806b514454f44022942c7ebc4abc5cd4 => generated 289 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:45.144185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7517f7-25c7-4f8a-99e1-9a1776390786 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.147449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.196957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.198176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.198507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.198805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.203927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.206060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.207876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 89/167] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:45.211823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.322351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7532a896-dd1e-4c32-9b2a-e9d5c5120e43 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.342438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.351091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.370361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.371050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.371327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.371596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.372841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.374302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.377359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3148eb-95b7-400c-9f1c-3387ad430b0f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.379606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 90/168] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:45.381133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.435679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.436314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.436586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.436849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.438014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.439486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.446860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 79/169] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:45.448475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.550200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06f000-633f-403c-a185-e9470db0df3f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.594653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74679-fc4c-4165-9a69-276cdd337f65 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.595953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.596746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.597843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.598120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.610598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:45.617064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:45.617946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:45.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 91/170] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:45.619544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.638996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.640460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.640787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.641355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.679179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.682797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.726986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.729367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884a6d-d579-4015-be57-f521c15c83ce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.737370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bcba0-1bdc-47ea-adae-ad73e44c1afd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.770950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.771678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.771951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.772217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/806b514454f44022942c7ebc4abc5cd4/roles/4b5bb1647c844291b8990ac8c47ba847` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.774105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, user_id=806b514454f44022942c7ebc4abc5cd4, role_id=4b5bb1647c844291b8990ac8c47ba847)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.775604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.790871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 92/171] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:45 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/users/806b514454f44022942c7ebc4abc5cd4/roles/4b5bb1647c844291b8990ac8c47ba847 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:45.798776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.849609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 80/172] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:45.851011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f553-31dc-4852-8fce-97f511c97bd8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.851345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f553-31dc-4852-8fce-97f511c97bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.851670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f553-31dc-4852-8fce-97f511c97bd8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.853285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 81/173] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:45.854184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f553-31dc-4852-8fce-97f511c97bd8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:45.915574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4c8e-584d-46c3-9d7b-28f26d36ef12 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.927239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.930608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:45.974069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:45.974917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.975291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.975654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.976513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:45.976928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:45.979603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f12e8f-3628-4462-a2f3-0080c0ca8575 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:45.988504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 93/174] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=d7498f66178947778670d4d051df8536 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:45.990287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755186-8607-4b56-9e8e-7c175c3894e2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:45.990714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755186-8607-4b56-9e8e-7c175c3894e2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:45.991078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755186-8607-4b56-9e8e-7c175c3894e2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:45.992976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f553-31dc-4852-8fce-97f511c97bd8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:45.997964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 94/175] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:45.999193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc755186-8607-4b56-9e8e-7c175c3894e2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.051125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.051497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.051791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.052026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.054782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:46.054782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:46.054782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1bdd8f-83b9-461b-9812-e7379a1f6c6d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:46.069304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 82/176] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:46.070972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9d78-8dbd-4e6a-b6b4-797e5dec92fb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.071475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9d78-8dbd-4e6a-b6b4-797e5dec92fb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.071975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f9d78-8dbd-4e6a-b6b4-797e5dec92fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.095110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b08f9d78-8dbd-4e6a-b6b4-797e5dec92fb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:46.096898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b08f9d78-8dbd-4e6a-b6b4-797e5dec92fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:46.140040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc755186-8607-4b56-9e8e-7c175c3894e2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.177225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 83/177] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:46.186838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9a54-5683-445f-b50c-02fe3f222a49 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.186838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9a54-5683-445f-b50c-02fe3f222a49 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.186838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df9a54-5683-445f-b50c-02fe3f222a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.193095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.193806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.194087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.194355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.196144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:46.197778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:46.200149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 95/178] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:45 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:46.200747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.213845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c4df9a54-5683-445f-b50c-02fe3f222a49 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:46.215373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c4df9a54-5683-445f-b50c-02fe3f222a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:46.262315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:46.274989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:46.303158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 84/179] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:46.304425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6a9d-5673-4ae1-adb4-0a5c7bc050e4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.304693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6a9d-5673-4ae1-adb4-0a5c7bc050e4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.304964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6a9d-5673-4ae1-adb4-0a5c7bc050e4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.306215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 85/180] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:46.311153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b6a9d-5673-4ae1-adb4-0a5c7bc050e4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.329256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb7b37-b835-458f-9a7e-3b210e4bbc60 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.395918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.396484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.396758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.397025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] PATH_INFO: `/v3/users/3b6af059a40d42aa92232a0f849bc581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.399510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] RBAC: Authorizing `identity:get_user(user_id=3b6af059a40d42aa92232a0f849bc581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:46.401093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145aa9-a1d9-41ac-be43-3cf8bcdc469d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:46.402942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 96/181] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:46 2022] GET /identity/v3/users/3b6af059a40d42aa92232a0f849bc581 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:46.403819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.404359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.406473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.436930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:46.438149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:46.480595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b6a9d-5673-4ae1-adb4-0a5c7bc050e4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.531997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.532604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.538836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.539144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.539907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:46.540215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:46.541702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:46.547956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 86/182] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:46 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:46.548918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.567030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 97/183] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:46.568383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.656011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:46.661626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:46.767067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79104-b18e-48f9-815c-d16f9f794cc0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.773749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2246dad-836f-4639-9ff3-b9283cd06a2e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.812850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.813607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.813960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.814545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.821314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:46.822258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:46.822774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:46.823241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:46.826647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:46.832583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:46.833477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:46.834267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 98/184] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:46.836190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:46.914205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:46.916690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:46.967649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:46.970932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc80531-2d87-46fb-b13d-5b0dd64b11d7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:46.972626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.006616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.007248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.007917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.007917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.018771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:47.023890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:47.024743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:47.026113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 99/185] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:47.027364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.044279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 87/186] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:47.044279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.136999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdedb9-a148-413b-8160-f7e25339ecb3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.142797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfb55b-5a49-4fcf-8b10-f72e02410884 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.170768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.171302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.171584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.171850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.172514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:47.172812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.174191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.179040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.179668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.179944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.180215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.181312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.182724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 100/187] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.183512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.188111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.193987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 88/188] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.193987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.206555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.207102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.284634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627056b2-6d51-49d6-87d2-ae0771e7e941 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.306353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d767b-c2f3-4f7c-82c6-073c65e6b9ec None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.318619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.319232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.319508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.319770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.320432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:47.320796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.322105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.331904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 101/189] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.332662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.348401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.349026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.349289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.349564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] PATH_INFO: `/v3/projects/8c0ab0e0d28445ab8116d115510e894a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.350694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] RBAC: Authorizing `identity:get_project(project_id=8c0ab0e0d28445ab8116d115510e894a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.352131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.354306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 89/190] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/projects/8c0ab0e0d28445ab8116d115510e894a => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.354306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.424071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.441201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.442781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eb40b-e431-4de6-868a-151ed40c5158 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.462540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29cfe4-dd27-41c8-b719-be80ad25b10b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.485717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.486347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.486644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.486915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.499444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.500067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.500311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.500577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.500689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.501218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.502128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.502667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.508542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 102/191] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.508542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.531195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 90/192] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:47 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:47.531195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.637207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eecac-3ca4-4388-9657-ad0a3ca6e532 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.642195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b275ec-0e4e-40a2-95fd-3cf1eb258d62 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.678464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.679048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.679323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.679590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.679765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.680324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=8c0ab0e0d28445ab8116d115510e894a, user.id=3b6af059a40d42aa92232a0f849bc581) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:47.680423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.680627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.680721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.680922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.682635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.682742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.684258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.684644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.687445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 103/193] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.688619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.699597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 91/194] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/role_assignments?user.id=3b6af059a40d42aa92232a0f849bc581&scope.project.id=8c0ab0e0d28445ab8116d115510e894a&role.id=d7498f66178947778670d4d051df8536 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:47.700344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:47.704566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.811019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd1bed-a1ba-4d3f-a49d-02c082935c95 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.815885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42531a01-fe2b-4df3-b6a0-bc5682d4099e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:47.851893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.852296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:47.852913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.853129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:47.853247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.853961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.854416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:47.854929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:47.930247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:47.932052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.933542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.933983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:47.935940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:47.946559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:48.065719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 104/195] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:48.067159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 92/196] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:48.071786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:48.073266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:48.195764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:48.201309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cb9bf-059f-405f-865d-0afb1ab86b73 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:48.207179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fc816-c1dd-4722-9ec9-083f4dbe5aa6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:48.221836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:48.250532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:48.251480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.251924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.252334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.252805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:48.253548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.253828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.254077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:48.254183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.255238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:48.255946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:48.256676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:48.257868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 93/197] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:48 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:48.261025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 105/198] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:48 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:48.263977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:48.266517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:48.392770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23907832-e22e-4397-8075-6be31ce8aa32 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:48.398393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df850e31-4735-484b-9e1b-25dbcec0ac54 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:48.429027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:48.429888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.430299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.430770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.431697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=91f47bce6d7e4440aeaff0034cbdf146) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:48.432196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:48.433750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8b63c-14bb-4f92-aeda-72cc5ffcfd3f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:48.439684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 94/199] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:48 2022] GET /identity/v3/role_assignments?user.id=91f47bce6d7e4440aeaff0034cbdf146&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:48.440872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2b79-e5d5-4e66-9736-3ace4f46235d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.441156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2b79-e5d5-4e66-9736-3ace4f46235d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.441445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2b79-e5d5-4e66-9736-3ace4f46235d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.442806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 95/200] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:48.443159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:48.448176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.448176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.448176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.448176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=91f47bce6d7e4440aeaff0034cbdf146) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:48.448526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:48.451950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:48.452121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:48.452379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:48.452651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:48.468287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 106/201] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:48 2022] GET /identity/v3/role_assignments?user.id=91f47bce6d7e4440aeaff0034cbdf146&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:48.488861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:48.490133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:48.544119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:48.609124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 96/202] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:49.003379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:49.053399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:49.122730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6e86e-22a2-4acd-aee8-b0dfc5597304 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:49.156166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:49.156798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:49.157072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:49.157342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:49.167435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:49.172659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:49.173513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:49.174965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 107/203] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:49.184025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:49.328270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc22a4-7528-4434-92d3-6ab6d58b9caa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:49.374068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:49.374705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:49.374987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:49.375260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:49.375931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:49.376231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:49.377607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:49.391294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 97/204] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:49.396974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:49.562282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323867b2-a83a-4b10-b4b9-447acc0cdf4d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:49.563814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:49.600931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:49.601479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:49.601753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:49.602068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] PATH_INFO: `/v3/users/806b514454f44022942c7ebc4abc5cd4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:49.603271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] RBAC: Authorizing `identity:get_user(user_id=806b514454f44022942c7ebc4abc5cd4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:49.605300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:49.607288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 108/205] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:49 2022] GET /identity/v3/users/806b514454f44022942c7ebc4abc5cd4 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:49.615512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:49.668078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:49.776219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd911f9-09e3-4a78-90a8-5429ec5df9a8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:49.820498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:49.821102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:49.821376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:49.821644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:49.905901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:49.907444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:49.933548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:50.027618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 98/206] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:50.033447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:50.186414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221085a-6b4e-41f7-b955-9247a3e24324 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:50.224757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:50.225388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.225664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.225933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.227074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:50.229037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135502e-692e-4092-a89d-3b0cb9e4c252 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:50.230798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 109/207] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:50 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:50.236744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:50.347710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd6d85-1bea-4b0a-8d12-d25f673d1fb4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:50.371096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:50.384266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:50.385383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.385655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.385918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.386660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, scope.project.id=0a5be0c65ca0402189e08b37aa717cca, user.id=806b514454f44022942c7ebc4abc5cd4) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:50.386961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:50.388371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a05c2-8a23-4eb8-844e-8fc9b4ac9ede None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:50.394599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 99/208] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:50 2022] GET /identity/v3/role_assignments?user.id=806b514454f44022942c7ebc4abc5cd4&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:50.544086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86477-6509-416e-a1c8-4357e851bc60 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.544771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86477-6509-416e-a1c8-4357e851bc60 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.545237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86477-6509-416e-a1c8-4357e851bc60 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.546816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 110/209] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:50.552631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a027c-2412-45e5-bcdd-e812a786b39e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.552925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a027c-2412-45e5-bcdd-e812a786b39e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.553176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a027c-2412-45e5-bcdd-e812a786b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.581527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4b7a027c-2412-45e5-bcdd-e812a786b39e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:50.582775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b7a027c-2412-45e5-bcdd-e812a786b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:50.674446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 100/210] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:50.850803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eec43-5cf6-48d1-ac25-d3c0c60c6e00 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.851434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eec43-5cf6-48d1-ac25-d3c0c60c6e00 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.851925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eec43-5cf6-48d1-ac25-d3c0c60c6e00 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.854027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 111/211] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:50.859613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:50.859886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:50.860159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:50.881123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:50.882603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:50.955004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eec43-5cf6-48d1-ac25-d3c0c60c6e00 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:50.963711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 101/212] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:50.998499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.070857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eec43-5cf6-48d1-ac25-d3c0c60c6e00 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.107317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.108329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.108806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.109270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.122247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bf44bb30-1bd1-4f5b-aae7-0e35c759ae20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:51.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:51.128846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:51.129658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:51.132323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 112/213] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:10:51.141005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.188391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.234726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ccd73-32a7-4346-8c0d-5b0502b5e3b7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.235382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ccd73-32a7-4346-8c0d-5b0502b5e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.235766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ccd73-32a7-4346-8c0d-5b0502b5e3b7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.237522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 113/214] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:51.247190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b2fd-6544-4cb2-a196-0ff18c73885c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.247869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b2fd-6544-4cb2-a196-0ff18c73885c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.248477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b2fd-6544-4cb2-a196-0ff18c73885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.249580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fb2a7-226b-4335-b9ed-9fe02a896ba7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.269185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c935b2fd-6544-4cb2-a196-0ff18c73885c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:51.270399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c935b2fd-6544-4cb2-a196-0ff18c73885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:51.287239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.287851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.288117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.288385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.289041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:51.289335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:51.290717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:51.298645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 102/215] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:51.298645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.344147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 114/216] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:51.345228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7cfef-10ac-4a94-aea9-79d4114164a7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.345507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7cfef-10ac-4a94-aea9-79d4114164a7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.345770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7cfef-10ac-4a94-aea9-79d4114164a7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.347084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 115/217] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:51.350758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7cfef-10ac-4a94-aea9-79d4114164a7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.380324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.408318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e31c5-b649-4cad-bef7-6310f3633c01 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.418993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.446316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.446964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.447299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.447565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.448676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:51.450120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce3080-a957-422c-945b-0110629e5a3a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:51.451964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 103/218] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:51.457343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.457657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.457916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.458614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7cfef-10ac-4a94-aea9-79d4114164a7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.475779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:51.476958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:51.497412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.498043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.498320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.498602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] PATH_INFO: `/v3/users/73c3a774f654498fadf3bf1d4343bded` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.512692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] RBAC: Authorizing `identity:get_user(user_id=73c3a774f654498fadf3bf1d4343bded)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:51.514137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:51.520538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 116/219] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/users/73c3a774f654498fadf3bf1d4343bded => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:51.520538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.548666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 104/220] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:51.558777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.607305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.622914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed440c-fdf6-48dd-bc51-fd5b482634eb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.657160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c1965-402b-4246-8702-33601e806999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.657763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c1965-402b-4246-8702-33601e806999 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.658037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c1965-402b-4246-8702-33601e806999 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.658303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c1965-402b-4246-8702-33601e806999 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.659034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c1965-402b-4246-8702-33601e806999 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:51.660411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c1965-402b-4246-8702-33601e806999 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:51.679382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ead766-c3e8-4e7a-aad8-9ca34471c9ba None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.683349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 117/221] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:51 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:51.688251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.691744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c1965-402b-4246-8702-33601e806999 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.739442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.740072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.740353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.740623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.820869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c1965-402b-4246-8702-33601e806999 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:51.840578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:51.842069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:51.859273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:51.860165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:51.860165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:51.860727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:51.874649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:51.880320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:51.881200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:51.881979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 118/222] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:51.882678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.891010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:51.961920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 105/223] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:51.962971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:51.972402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.008430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e88d6-dd31-41c8-80db-153251853d03 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.048883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.049522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.049795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.050058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.061494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:52.067977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:52.068857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:52.069625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 119/224] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:52.070266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.103997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fdbd1-a117-474f-8f84-12f92d307192 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.144535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.152965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.154143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.154672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.155307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.157054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:52.157621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.159214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.171374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 106/225] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.172603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.191505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5363-0f51-47d0-b000-8d32cab47829 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.236685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.237349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.237692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.238030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.239238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.241541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.244768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 120/226] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.245874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.260293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.305574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ab4a-ffb6-4745-9459-6f41d1fdebd4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.345493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.346394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.347040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.347506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.348820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:52.349365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.351085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.361161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 107/227] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.362369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.368707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41c6c-d8df-46f9-8305-a0dff3a58d16 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.404628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.412319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.413077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.413375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.413763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.414924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.417449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.419515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 121/228] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.420256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.476542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5988e6-a405-4b67-aaad-4b40fc010683 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.521282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.529837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.530448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.530755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.531025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.532116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=73c3a774f654498fadf3bf1d4343bded) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:52.532497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.534612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.542369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 108/229] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/role_assignments?user.id=73c3a774f654498fadf3bf1d4343bded&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=d7498f66178947778670d4d051df8536 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.543612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.551805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1bd20-4aff-426b-b04e-f227ac377c2c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.592135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.593390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.593900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.594376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.595943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.597693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.599815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 122/230] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.611416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.676417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48691c0-b5cf-44db-9e4b-f574b23b223f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.725261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.725890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.726160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.726429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.731489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.734114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.741108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 109/231] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.746463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.772805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.813268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8ea45-51c4-4a44-acbb-9f871b170318 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.822068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:52.879513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.880161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.880438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.880710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.881831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.883389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.885538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 123/232] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:52 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:52.891813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:52.904541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd04170-a934-4167-8bed-a867ad57d0bb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:52.972361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:52.973024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:52.973299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:52.973568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/users/91f47bce6d7e4440aeaff0034cbdf146/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:52.977174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, user_id=91f47bce6d7e4440aeaff0034cbdf146, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:52.979248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7d97a-dd5d-42b4-8d1c-5862084a1a29 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:52.999816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 110/233] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:52 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/users/91f47bce6d7e4440aeaff0034cbdf146/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:10:53.037769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a5a0-d83c-4bf0-9a60-c659eea4403e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:53.081325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:53.082057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:53.082340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:53.082626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/users/91f47bce6d7e4440aeaff0034cbdf146/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:53.084876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, user_id=91f47bce6d7e4440aeaff0034cbdf146, role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:53.086333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1e386-6c3f-4451-b055-106f4c5018f1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:53.089330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:53.110396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 124/234] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:52 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/users/91f47bce6d7e4440aeaff0034cbdf146/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:54.183763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710abfd-2a76-4d9b-aca3-717978483bfc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.184312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710abfd-2a76-4d9b-aca3-717978483bfc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.184312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710abfd-2a76-4d9b-aca3-717978483bfc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.185644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 111/235] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:54.191129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53217-a4f4-4fda-997f-581892cf616d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.191527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53217-a4f4-4fda-997f-581892cf616d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.191706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53217-a4f4-4fda-997f-581892cf616d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.215586 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dbe53217-a4f4-4fda-997f-581892cf616d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:54.216812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dbe53217-a4f4-4fda-997f-581892cf616d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:54.307609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 125/236] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:54.530250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05c88-4c62-4b61-af2f-7771c4725285 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.530521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05c88-4c62-4b61-af2f-7771c4725285 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.531080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05c88-4c62-4b61-af2f-7771c4725285 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.532609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 112/237] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:54.538050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.538340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.538623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.567520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:54.569039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:54.618007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05c88-4c62-4b61-af2f-7771c4725285 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:54.657572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 126/238] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:54.766562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05c88-4c62-4b61-af2f-7771c4725285 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:54.798938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:54.809953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:54.810624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.811375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.812049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.831450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:54.836265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:54.837111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:54.838323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 113/239] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:10:54.843365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:54.904819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:54.950733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b824-1b59-4dbe-a9ab-41d9c4dea4e4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:54.978063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:54.989141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:54.989756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:54.990034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:54.990307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:54.991011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:54.991319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:54.998597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.001723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 127/240] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:55.007700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:55.172220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eadbb-65eb-4f4e-8e96-aa3b0a142635 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:55.188024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188643ef-3612-4576-ac33-8b88e4f8a2a1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:55.207826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:55.238124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:55.239198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.239483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.239753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] PATH_INFO: `/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.244615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:55.245466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.245933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.246393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.247710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:55.249363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8aaba-aea5-49dc-8a26-4887fc27af13 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.251170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 114/241] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:55.256539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] RBAC: Authorizing `identity:get_user(user_id=fcb5fecb96bd4b8f906ee7245c42ee1f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:55.258011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.258892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822dad8-16cd-451b-90f8-648475d7ab1a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.259449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822dad8-16cd-451b-90f8-648475d7ab1a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.260110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822dad8-16cd-451b-90f8-648475d7ab1a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.260452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 128/242] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:55 2022] GET /identity/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:55.261914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:55.262219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 115/243] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:55.269114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822dad8-16cd-451b-90f8-648475d7ab1a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:55.324998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:55.376969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822dad8-16cd-451b-90f8-648475d7ab1a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:55.380928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1df369-0ae8-457f-8103-bf1518df24be None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:55.412039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:55.412642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.413012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.413237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.419606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:55.420580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.421074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.421540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.422439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:55.424164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4492f-cfdd-486a-a2a4-6ff08be364c9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.456028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 129/244] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:55 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:55.457644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.458189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.465593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.488425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:55.489995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:55.522565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:55.535961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:55.537703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4b7fb-435a-4f8c-8685-f98be51fcdf3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.578448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 130/245] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:55.630108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:55.714675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 116/246] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:55.716589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:55.785394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8dbde-0ea5-48b1-af2a-908cf9654385 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.786034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8dbde-0ea5-48b1-af2a-908cf9654385 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.786505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8dbde-0ea5-48b1-af2a-908cf9654385 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.788644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 117/247] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:55.797638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b876e-6de7-4428-8275-cc3715362b07 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.797638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b876e-6de7-4428-8275-cc3715362b07 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.797976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b876e-6de7-4428-8275-cc3715362b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.818791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-876b876e-6de7-4428-8275-cc3715362b07 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:55.819979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-876b876e-6de7-4428-8275-cc3715362b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:55.851556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea808d7-2512-4e67-bb9b-d78ebe8b5c4a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:55.904303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 118/248] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:55.905588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:55.905752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9caad-eb50-49fd-96c3-2c80dbe3ec03 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.905976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9caad-eb50-49fd-96c3-2c80dbe3ec03 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.906235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.906330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a9caad-eb50-49fd-96c3-2c80dbe3ec03 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.906513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.906830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.907964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:55.909418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:55.911108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 131/249] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:55 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:55.912081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 119/250] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:55.917947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:55.922536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:55.922850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:55.923120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:55.945035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:55.946521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:55.958769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.034687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 120/251] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:56.034687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.045047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.064736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6838-840a-4140-8cfe-03f286a1314a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.111219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.112312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.112593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.112866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.113985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=fcb5fecb96bd4b8f906ee7245c42ee1f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:56.114300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:56.116708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:56.124401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 132/252] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:55 2022] GET /identity/v3/role_assignments?user.id=fcb5fecb96bd4b8f906ee7245c42ee1f&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:56.159285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fca50e-184e-4bfe-9a47-6d62a8033c3e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.197827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.198616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.198781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.199051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.206799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.211801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:56.218239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:56.219250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:56.222093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 121/253] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:56.227613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.317919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.346267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50801f-8dc2-4a9b-8539-dd2ae3796936 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.363787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.385243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.385852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.386125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.386391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.387089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:56.387384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:56.388020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef393b1e-8e99-45b5-81c8-85bbb71c674b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.388756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:56.400508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 122/254] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:56.403247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.438047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.438722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.439001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.439329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.452172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:56.457774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:56.458948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:56.460438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 133/255] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:56.464996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.549455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5923dc4-a89a-4366-bcba-f6789328d49a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.585314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.599322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.600103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.600378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.600648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.620695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:56.626794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:56.627803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 123/256] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:56.628830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.652687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb7038-d7e3-4af6-b8c2-2e726d2d1425 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.716580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.717786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.718280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.718853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] PATH_INFO: `/v3/users/73c3a774f654498fadf3bf1d4343bded` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.721774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] RBAC: Authorizing `identity:get_user(user_id=73c3a774f654498fadf3bf1d4343bded)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:56.723607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:56.725458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 134/257] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/users/73c3a774f654498fadf3bf1d4343bded => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:56.726646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.740022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641f8be-e0cf-4371-afa9-41dfd962d2e6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.764619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:56.810765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.811804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.812465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.812951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.813808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:56.814313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:56.816350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:56.827647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 124/258] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:56.828863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.905226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f37a7a-71a5-4ca5-86b0-184f52729389 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:56.941897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:56.942963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:56.943515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:56.943977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:56.944837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:56.945498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:56.947308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:56.956405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 135/259] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:56.957766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:56.976177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:57.074179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d308bb-25f0-489e-a360-c225acee64cc None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.102971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:57.127014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a157e3-9169-409d-888d-24250efb0adb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.143619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.144274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.144549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.144817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.146947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.148437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.152590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 125/260] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:57.153482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:57.198932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.199913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.200403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.200900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.204618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.206731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.208754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 136/261] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:56 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:57.209332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:57.300673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc954f1-3480-43c8-8231-6db52c22aea2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.325253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:57.336300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.337100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.337484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.337974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] PATH_INFO: `/v3/users/91f47bce6d7e4440aeaff0034cbdf146` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.339181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] RBAC: Authorizing `identity:get_user(user_id=91f47bce6d7e4440aeaff0034cbdf146)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.340692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.343695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 126/262] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:57 2022] GET /identity/v3/users/91f47bce6d7e4440aeaff0034cbdf146 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:57.344510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:57.388851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38361f-8313-42ee-9a61-27966bc4d19a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.463389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.463980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.464263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.464541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/users/73c3a774f654498fadf3bf1d4343bded/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.466538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, user_id=73c3a774f654498fadf3bf1d4343bded, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.468026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.500765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 137/263] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:10:57 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/users/73c3a774f654498fadf3bf1d4343bded/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:10:57.501582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:57.527503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:57.529022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f773ac0-057b-48bb-8ee2-0fc208bd74ef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.577506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.578403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.578886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.579310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.690633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:57.723753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.725170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.739777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e1836-23f5-44a4-8ff2-ca6ef8be4169 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:57.823341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:57.823971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:57.824251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:57.824524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:57.934546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 127/264] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:57.940453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:57.951999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:57.953482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:57.971795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:58.119561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:58.137966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b45f3-6fe9-4777-b463-6d98133ff406 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:58.197640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 138/265] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:10:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:10:58.203410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:58.227880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:58.228614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:58.228893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:58.229160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:58.233775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:58.235289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:58.236939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 128/266] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:57 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:58.243134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:58.380052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fb93e-c265-491c-8f65-8383aefc5cb0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:58.403312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:58.422641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a77d17-52f9-4353-be71-b625a9b2f3aa None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:58.444171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:58.445144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:58.445619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:58.446083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:58.451950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:58.453984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:58.454812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 139/267] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:10:58 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:58.464418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:58.496728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:58.497332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:58.497606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:58.497872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:58.498618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=91f47bce6d7e4440aeaff0034cbdf146) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:58.498977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:58.500711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232449f2-4f61-43dc-996d-a8acb8885e49 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:58.515629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 129/268] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:58 2022] GET /identity/v3/role_assignments?user.id=91f47bce6d7e4440aeaff0034cbdf146&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:10:58.628797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f0ba-e39b-4442-b92b-a0063221a64e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:58.673513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:58.674144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:58.674419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:58.674707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:58.675446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=91f47bce6d7e4440aeaff0034cbdf146) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:58.675748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:58.677194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbca9bb-fe0f-4095-89b0-6361773d5792 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:58.684126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 140/269] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:10:58 2022] GET /identity/v3/role_assignments?user.id=91f47bce6d7e4440aeaff0034cbdf146&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:58.991583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085f7-13c5-49c7-a625-1dc2798f63dc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:58.992258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085f7-13c5-49c7-a625-1dc2798f63dc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:58.992874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776085f7-13c5-49c7-a625-1dc2798f63dc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:58.995343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 130/270] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:59.000829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e950-1846-4f45-9429-2042a5da3510 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.001416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e950-1846-4f45-9429-2042a5da3510 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.001904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5e950-1846-4f45-9429-2042a5da3510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.027183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c0-9dbd-44e5-8e4a-024275d383b3 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.027828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c0-9dbd-44e5-8e4a-024275d383b3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.028100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319425c0-9dbd-44e5-8e4a-024275d383b3 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.029575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 131/271] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:59.032346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6cc5e950-1846-4f45-9429-2042a5da3510 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:59.033618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6cc5e950-1846-4f45-9429-2042a5da3510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:59.034970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.035242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.035505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.057846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:59.059821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:59.083276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.084330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.129197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 141/272] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:59.130522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59059a29-a4d1-48e8-9f11-1d9ba45aea7f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.130980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59059a29-a4d1-48e8-9f11-1d9ba45aea7f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.131252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59059a29-a4d1-48e8-9f11-1d9ba45aea7f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.132620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 142/273] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:10:59.142189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.142475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.142770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.164721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:10:59.166772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:10:59.167161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 132/274] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:10:59.242022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 143/275] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:10:59.420063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.451961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.552639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c3241-5ad6-43da-9744-0f87091fa814 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.559525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fbf7e-d4fb-45a4-a243-b193265f8989 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.561241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.588221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.588849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.589124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.589436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.598182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.599004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.599430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:59.599559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.599949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:59.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:59.605415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 133/276] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:59.606382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.618556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:10:59.624342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:10:59.626246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:10:59.627175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 144/277] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:10:59.629191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.717755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56adca7-e6ce-4c15-828b-a14dec10eb87 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.733551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f1c7c1-e6e4-463c-8d9b-069f2069dad6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.756541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.757162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.757436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.757702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.758831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:59.760340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:59.761905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 134/278] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:59.762673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.772798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.773773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.774052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.774319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.776006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:59.776247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:59.777569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:59.789912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 145/279] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:59.789912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.811144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.859367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:10:59.879968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a338f-1803-49c7-8791-3f6cde40b382 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.890336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a2fe9-1e86-4caf-906c-6e04360cf8e5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:10:59.921329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.921932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.922218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.922482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.923175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:10:59.923470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:59.924693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:10:59.924838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:59.925568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:10:59.925996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:10:59.926263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:10:59.927197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:10:59.928393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:10:59.933572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 135/280] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:10:59.937181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:10:59.954630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 146/281] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:10:59 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:10:59.954819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:00.060285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:00.083076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f78598-28f1-4a8f-8f04-7f5584bcb422 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.085302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aad89d-7b67-4e7e-a5b5-d3f9fae53dea None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.125240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.126134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.126610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.127024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] PATH_INFO: `/v3/users/25b979dc09e44a2a80268ce6ba1dee66` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.128116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.128865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.129239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.129610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] PATH_INFO: `/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.132335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb5fecb96bd4b8f906ee7245c42ee1f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:00.133934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:00.136570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 136/282] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:00.142039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:00.148979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] RBAC: Authorizing `identity:get_user(user_id=25b979dc09e44a2a80268ce6ba1dee66)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:00.150649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:00.152482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 147/283] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:10:59 2022] GET /identity/v3/users/25b979dc09e44a2a80268ce6ba1dee66 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:00.158503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:00.160205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:00.273531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf2416-ea45-41c0-be76-39ae730ce159 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.273820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384643-ca67-4d50-823f-625339aafb36 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.315417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.316054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.316331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.316600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.317689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:00.318569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.319211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.319312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59a228-5518-4554-98a1-3e0bd67ab2fa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:00.319480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.319753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.320487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, user.id=25b979dc09e44a2a80268ce6ba1dee66, scope.system=all) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:00.320796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:00.320921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 137/284] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:00 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:00.322561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5390-a8c1-4429-8621-75186c20d781 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.322946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5390-a8c1-4429-8621-75186c20d781 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.323269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5390-a8c1-4429-8621-75186c20d781 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.324462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 138/285] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:00.324855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1d753-0bde-4a7c-bd9f-8ac324f43fa9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:00.328163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b5390-a8c1-4429-8621-75186c20d781 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:00.332504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 148/286] 10.222.0.20 () {56 vars in 1308 bytes} [Sun Sep 18 06:11:00 2022] GET /identity/v3/role_assignments?user.id=25b979dc09e44a2a80268ce6ba1dee66&scope.system=all&role.id=d7498f66178947778670d4d051df8536 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:00.333722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.333910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.334164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.348292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:00.355209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:00.357252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:00.434763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:00.461559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 149/287] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:00.470611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b5390-a8c1-4429-8621-75186c20d781 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.534425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.535260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.535546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.535824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/users/fcb5fecb96bd4b8f906ee7245c42ee1f/roles/4b5bb1647c844291b8990ac8c47ba847` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.542523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, user_id=fcb5fecb96bd4b8f906ee7245c42ee1f, role_id=4b5bb1647c844291b8990ac8c47ba847)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:00.544134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:00.569433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 139/288] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:00 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/users/fcb5fecb96bd4b8f906ee7245c42ee1f/roles/4b5bb1647c844291b8990ac8c47ba847 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:11:00.793757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:00.903318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103749b-f1d9-43f8-9cc5-63043d45ac21 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:00.935804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:00.942863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:00.943689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:00.945443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:00.945748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:00.957467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:00.962366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:00.963366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:00.964227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 150/289] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:00.969562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:01.073936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5f672-ba88-49aa-b0ad-df13575872c1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:01.109408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:01.110020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:01.110300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:01.110586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:01.111272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:01.111571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:01.112954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:01.118262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:01.122400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 140/290] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:01.128403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:01.277070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f419a5-811b-403f-aa6c-26a9da5aec13 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:01.323290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:01.323932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:01.324268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:01.324484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] PATH_INFO: `/v3/users/73c3a774f654498fadf3bf1d4343bded` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:01.325663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] RBAC: Authorizing `identity:get_user(user_id=73c3a774f654498fadf3bf1d4343bded)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:01.327222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:01.328827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 151/291] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:01 2022] GET /identity/v3/users/73c3a774f654498fadf3bf1d4343bded => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:01.334496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:01.455116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d86bc-da56-4361-87e6-fe55d5725ac6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:01.491503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:01.492113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:01.492399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:01.492675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:01.559803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:01.566927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:01.568422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:01.716500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 141/292] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:01.722098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:01.800527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:01.843944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bf6a9-1f50-4638-b745-31678ba8c1de None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:01.894089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:01.894822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:01.895105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:01.895431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:01.896640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:01.898100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ced0cf-75f9-4562-a4a3-dfd32ba90006 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:01.899778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 152/293] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:01 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:01.906333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:02.035918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:02.039256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1a32f-26ba-481f-b635-8e00e6008f8d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:02.093091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:02.093701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.094028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.094307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.095069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=73c3a774f654498fadf3bf1d4343bded) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:02.095376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:02.096915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b372b-3999-4583-be18-cc26fafa54ff None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:02.107241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 142/294] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:01 2022] GET /identity/v3/role_assignments?user.id=73c3a774f654498fadf3bf1d4343bded&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=d7498f66178947778670d4d051df8536 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:02.682722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f54e2-aca7-41ec-881f-7f63dd34a9d2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.683087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f54e2-aca7-41ec-881f-7f63dd34a9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.683369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f54e2-aca7-41ec-881f-7f63dd34a9d2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.685186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 153/295] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:02.686222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6e89-2986-448f-bf3c-97f4e80da5a3 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.686505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6e89-2986-448f-bf3c-97f4e80da5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.686809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe6e89-2986-448f-bf3c-97f4e80da5a3 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.688135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 143/296] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:02.691592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3198f2-36e8-479f-9327-c9ec4cfd230e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.691876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3198f2-36e8-479f-9327-c9ec4cfd230e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.692195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3198f2-36e8-479f-9327-c9ec4cfd230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.694037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57b48-8513-47b3-9f2e-cdba9860e815 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.694334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57b48-8513-47b3-9f2e-cdba9860e815 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.694617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57b48-8513-47b3-9f2e-cdba9860e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.716064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41a57b48-8513-47b3-9f2e-cdba9860e815 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:02.717197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41a57b48-8513-47b3-9f2e-cdba9860e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:02.717735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-db3198f2-36e8-479f-9327-c9ec4cfd230e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:02.718623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db3198f2-36e8-479f-9327-c9ec4cfd230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:02.800365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 144/297] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:02.817355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 154/298] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:02.992457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4ae26-c640-482d-abdb-87e2f057d3f7 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:02.992747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4ae26-c640-482d-abdb-87e2f057d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:02.993027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4ae26-c640-482d-abdb-87e2f057d3f7 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:02.994699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 145/299] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:02.999737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.000020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.000291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.019107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:03.020283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:03.080229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.083463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4ae26-c640-482d-abdb-87e2f057d3f7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.102945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 155/300] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:03.106755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.207971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.218134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4ae26-c640-482d-abdb-87e2f057d3f7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.238916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7947b21-1662-4676-9b5b-7a89c607724e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.262891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.263428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.263704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.264023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.265533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.267045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.269291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 146/301] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:03.274260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.283702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.284594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.285080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.285554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.296280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:03.301425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:03.302314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:03.303107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:03.310640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 156/302] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:03.311780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.363557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.396659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfeb68b-01fc-4b76-8762-899f33366b2b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.420675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae1006-649e-4bcc-80c2-be57b4f76d24 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.432539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.433139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.433414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.433683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] PATH_INFO: `/v3/groups` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.434305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.435770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.454969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.455576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.455849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.456114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.456791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:03.457074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.457165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 147/303] 10.222.0.20 () {60 vars in 1134 bytes} [Sun Sep 18 06:11:03 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:11:03.458759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.467829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.476718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 157/304] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:03.483095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.505794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.593673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a4745-f988-49f8-a919-0da3100416eb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.626686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76918b84-eafb-4e04-8b56-8d0e48b891d2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.631408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.632049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.632321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.632589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.638877 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.645492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-cb8c25f8-505c-45da-bf31-dec673ed3ba9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:03.651683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:03.652544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 148/305] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:03.654750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79b78-0157-4564-bfd9-7f250442efa9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.655457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79b78-0157-4564-bfd9-7f250442efa9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.655457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e79b78-0157-4564-bfd9-7f250442efa9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.656216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 149/306] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:03.660959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e79b78-0157-4564-bfd9-7f250442efa9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.670659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.671308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.671576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.671847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] PATH_INFO: `/v3/users/6ab7d3155489419d95e095c65db6f24a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.686825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab7d3155489419d95e095c65db6f24a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.688301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73812dd-48ef-4e91-b15f-4655bc324640 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.690015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 158/307] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/users/6ab7d3155489419d95e095c65db6f24a => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:03.691146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.691415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.691684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.709372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:03.710567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:03.761021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.783029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e79b78-0157-4564-bfd9-7f250442efa9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.784288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 159/308] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:03.785239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.827760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.828530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.828887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.829316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.830052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:03.830610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.832160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.847865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 150/309] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:03.852822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:03.911052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:03.917172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef84eb9-3c02-4382-bc62-6a4bf219c327 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:03.954591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:03.955339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:03.955684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:03.956069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:03.956906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, user.id=6ab7d3155489419d95e095c65db6f24a, scope.system=all) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:03.957473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:03.958960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:03.965345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 160/310] 10.222.0.20 () {56 vars in 1308 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/role_assignments?user.id=6ab7d3155489419d95e095c65db6f24a&scope.system=all&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:03.967752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9940afc-fee3-422b-a6c3-c54e42959c4f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.011432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.012376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.012863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.013310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] PATH_INFO: `/v3/users/25b979dc09e44a2a80268ce6ba1dee66` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.014730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] RBAC: Authorizing `identity:get_user(user_id=25b979dc09e44a2a80268ce6ba1dee66)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:04.016630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:04.018497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 151/311] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:03 2022] GET /identity/v3/users/25b979dc09e44a2a80268ce6ba1dee66 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:04.028210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:04.066126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:04.085736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:04.147024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670aed44-c34c-4165-bd7e-5d06093b0b03 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.197348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cd2c-5906-45f7-8c10-0441678768ef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.200672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.201337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.201611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.201882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] PATH_INFO: `/v3/system/users/25b979dc09e44a2a80268ce6ba1dee66/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.203345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=25b979dc09e44a2a80268ce6ba1dee66, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:04.204721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:04.221799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 161/312] 10.222.0.20 () {58 vars in 1334 bytes} [Sun Sep 18 06:11:04 2022] PUT /identity/v3/system/users/25b979dc09e44a2a80268ce6ba1dee66/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:04.236021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.236711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.236963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.237342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.249112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:04.254287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:04.255273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:04.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:04.260759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 152/313] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:04.264004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:04.300022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:04.390339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2ebd1-d9d0-4b58-99f0-902931aff859 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.429219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.429782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.430059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.430335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.431054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:04.431412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:04.432801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:04.441729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 162/314] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:04.448618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:04.533768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:04.565348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3c02-61a4-4a17-af5a-e1850b21c330 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.600275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.600896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.601166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.601429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] PATH_INFO: `/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.602532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] RBAC: Authorizing `identity:get_user(user_id=fcb5fecb96bd4b8f906ee7245c42ee1f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:04.604523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:04.606460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 153/315] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:04 2022] GET /identity/v3/users/fcb5fecb96bd4b8f906ee7245c42ee1f => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:04.611729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:04.703444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:04.717248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f4594-4a25-4055-8f41-dcfce765c6a7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:04.753839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:04.754458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:04.754770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:04.755136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:04.823503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:04.825034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:04.922334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:04.942317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 163/316] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:04.943669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:05.067849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c04459-b729-41b8-9a0c-a390f46111b7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:05.103488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:05.104108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:05.104378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:05.104639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:05.105748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:05.107203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611e220-6617-4f0a-aa33-ba3fa501a7bd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:05.109106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 154/317] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:04 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:05.114786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:05.217359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd33993-be8a-41cf-a36d-b29c31e924c1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:05.257002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:05.257966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:05.258475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:05.258972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:05.260499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, scope.project.id=d47864572a6a4628875eb3a75e820751, user.id=fcb5fecb96bd4b8f906ee7245c42ee1f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:05.261008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:05.262673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bde585-a725-40d3-83ff-8a22e9d47dcd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:05.269495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 164/318] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:05 2022] GET /identity/v3/role_assignments?user.id=fcb5fecb96bd4b8f906ee7245c42ee1f&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:05.815480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e0c3-d81c-4415-a3df-14e92592d827 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:05.815675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e0c3-d81c-4415-a3df-14e92592d827 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:05.815960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50e0c3-d81c-4415-a3df-14e92592d827 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:05.817512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 155/319] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:05.822769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ff24-9c43-457a-b267-a57ddc1f7cf2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:05.823054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ff24-9c43-457a-b267-a57ddc1f7cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:05.823322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ff24-9c43-457a-b267-a57ddc1f7cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:05.841676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6c76ff24-9c43-457a-b267-a57ddc1f7cf2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:05.842914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6c76ff24-9c43-457a-b267-a57ddc1f7cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:05.891329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:05.912179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 165/320] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:06.179256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50e0c3-d81c-4415-a3df-14e92592d827 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:06.190384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:06.205181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eade6-883f-43db-bb18-a74576740017 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.205662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eade6-883f-43db-bb18-a74576740017 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.205916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eade6-883f-43db-bb18-a74576740017 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.207619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 166/321] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:06.213184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.213463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.213731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.231148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:06.232342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:06.288089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50e0c3-d81c-4415-a3df-14e92592d827 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:06.302808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 167/322] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:06.323528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:06.324151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.324422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.324689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.325767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:06.327225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156e29a-c3ca-4330-872d-b6d6f900d8ce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:06.328922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 156/323] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:06.334368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:06.439192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df60fd-dadd-4f11-ba66-6457f35f4c98 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:06.454816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:06.479641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:06.480253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.480620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.480895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] PATH_INFO: `/v3/groups` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.481525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:06.482979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e2ca4-c881-4225-adf0-ce56e12862e6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:06.506614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 168/324] 10.222.0.20 () {60 vars in 1134 bytes} [Sun Sep 18 06:11:06 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:06.520136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01564f0-4632-4e1e-aa62-5875717f5aea None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.520439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01564f0-4632-4e1e-aa62-5875717f5aea None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.520710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01564f0-4632-4e1e-aa62-5875717f5aea None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.522348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 157/325] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:06.528650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.528957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.529194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.548376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:06.549586 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:06.578715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01564f0-4632-4e1e-aa62-5875717f5aea None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:06.626303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 169/326] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:06.668857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:06.697006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01564f0-4632-4e1e-aa62-5875717f5aea None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:06.736721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:06.737360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.737638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.737911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.749428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:06.754288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:06.755219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:06.756016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 158/327] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:06.761363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:06.875854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e013d-6b3b-4223-9f97-ab25b840f7cc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:06.877620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:06.900978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:06.912227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:06.913403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:06.913584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:06.913835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:06.914514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:06.914844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:06.917792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:06.928852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 170/328] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:06.933860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:07.034696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5813d5-0adc-45f9-bc12-a8973b14694a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.064711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f8a03-48e1-458a-b9b5-e6b0be058606 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.080128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.081102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.081669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.082145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.085464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:07.097823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:07.104154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:07.105064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:07.105839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:07.108596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 159/329] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:07.113126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.114032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.114534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:07.114661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.115211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] PATH_INFO: `/v3/users/6ab7d3155489419d95e095c65db6f24a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.116593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab7d3155489419d95e095c65db6f24a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:07.118253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:07.120567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 171/330] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:06 2022] GET /identity/v3/users/6ab7d3155489419d95e095c65db6f24a => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:07.126337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:07.162040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:07.243432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f468c-7ef6-439e-9008-97eca235ebe7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.249186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0da74-cfbb-4d36-a081-147cac8a8999 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.291637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.292034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e1e65-b617-4615-8944-942586651f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.292150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.292420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.292691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.293354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:07.293650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:07.293867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1e65-b617-4615-8944-942586651f62 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.294341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1e65-b617-4615-8944-942586651f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.294841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e1e65-b617-4615-8944-942586651f62 None admin] PATH_INFO: `/v3/system/users/6ab7d3155489419d95e095c65db6f24a/roles/4b5bb1647c844291b8990ac8c47ba847` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.295047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:07.296655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e1e65-b617-4615-8944-942586651f62 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6ab7d3155489419d95e095c65db6f24a, role_id=4b5bb1647c844291b8990ac8c47ba847)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:07.298235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e1e65-b617-4615-8944-942586651f62 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:07.308884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 160/331] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:07.312228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:07.312564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 172/332] 10.222.0.20 () {58 vars in 1334 bytes} [Sun Sep 18 06:11:07 2022] PUT /identity/v3/system/users/6ab7d3155489419d95e095c65db6f24a/roles/4b5bb1647c844291b8990ac8c47ba847 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:11:07.381985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:07.430704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76e30b-2222-4590-9326-09fdbc4e47ca None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.467939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.468381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.468663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.468939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] PATH_INFO: `/v3/users/25b979dc09e44a2a80268ce6ba1dee66` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.470194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] RBAC: Authorizing `identity:get_user(user_id=25b979dc09e44a2a80268ce6ba1dee66)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:07.471727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6f772-94bc-4d9b-84dd-8bd56cb08e7c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:07.473751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 161/333] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:07 2022] GET /identity/v3/users/25b979dc09e44a2a80268ce6ba1dee66 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:07.480285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e1e65-b617-4615-8944-942586651f62 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:07.592232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:07.600539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e1e65-b617-4615-8944-942586651f62 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:07.640297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:07.640920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:07.641197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:07.641465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:07.642199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] RBAC: Adding query filter params (role.id=d7498f66178947778670d4d051df8536, user.id=25b979dc09e44a2a80268ce6ba1dee66, scope.system=all) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:07.642504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:07.643976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafd7c8-db55-4145-a583-b26e71a59c1c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:07.657818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 173/334] 10.222.0.20 () {56 vars in 1308 bytes} [Sun Sep 18 06:11:07 2022] GET /identity/v3/role_assignments?user.id=25b979dc09e44a2a80268ce6ba1dee66&scope.system=all&role.id=d7498f66178947778670d4d051df8536 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:09.235917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4097-b000-44ee-925c-c1f3c546f9aa None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:09.236454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4097-b000-44ee-925c-c1f3c546f9aa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:09.236533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4097-b000-44ee-925c-c1f3c546f9aa None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:09.238197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 162/335] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:09.245663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449a295-1e66-4fb4-bbfa-16a1857209b0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:09.246160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449a295-1e66-4fb4-bbfa-16a1857209b0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:09.246257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449a295-1e66-4fb4-bbfa-16a1857209b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:09.269363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d449a295-1e66-4fb4-bbfa-16a1857209b0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:09.270618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d449a295-1e66-4fb4-bbfa-16a1857209b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:09.350770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 174/336] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:09.418047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e284-577e-41f7-ab45-1a5f8bc5c6f0 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:09.418701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e284-577e-41f7-ab45-1a5f8bc5c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:09.419196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7e284-577e-41f7-ab45-1a5f8bc5c6f0 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:09.420953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 163/337] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:09.429450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:09.429731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:09.429993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:09.453872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:09.455107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:09.540463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 175/338] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:09.800429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e284-577e-41f7-ab45-1a5f8bc5c6f0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:09.932288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7e284-577e-41f7-ab45-1a5f8bc5c6f0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:09.961273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:09.975728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:09.976364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:09.976638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:09.976908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:09.987569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:09.992992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:09.994398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:09.995236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:09.999013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 164/339] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:10.005859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.028727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.035642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:10.210046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eba2b6-f3b9-4912-aa37-564c30e9f66d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:10.213428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30c69e-9f6d-4ba6-b590-75b59eb21495 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:10.260563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:10.262998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.264026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.264657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.271665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:10.275591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.276098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.276827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.278358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:10.279074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:10.281600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:10.288232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:10.297993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:10.298969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:10.299767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:10.301540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 165/340] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:10.307587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.309221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 176/341] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:10.322451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.385824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:10.443734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:10.461875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee241f4d-84c8-410e-90a7-42e0d3febfce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:10.484613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371e2ee-6c00-41e1-a6f0-b9b4bb3fdf7d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:10.505725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:10.506333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.506894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.507018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.507644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:10.507942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:10.509834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798658d9-bce2-433e-9b69-1acbb69bdc66 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:10.520604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 166/342] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:10.521844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f766-71fb-4b5b-8a75-3ff3af0c7ae9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.522120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f766-71fb-4b5b-8a75-3ff3af0c7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.522389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f766-71fb-4b5b-8a75-3ff3af0c7ae9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.523879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 167/343] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:10.528296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2f8-07e3-4af4-8687-ecb7202f14b4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.528296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2f8-07e3-4af4-8687-ecb7202f14b4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.528296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2f8-07e3-4af4-8687-ecb7202f14b4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.528296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 168/344] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:10.528603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b2f8-07e3-4af4-8687-ecb7202f14b4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.536905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:10.537848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.538329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.538908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.613746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:10.615606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c3f4b-3910-4905-a7bf-bbd52d73df95 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:10.648124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b2f8-07e3-4af4-8687-ecb7202f14b4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:10.663681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:10.689517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:10.690453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.690983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.691395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.706773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:10.734417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 177/345] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 06:11:10.736173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23010-0e94-4726-8f29-e43749fb4234 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.736669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23010-0e94-4726-8f29-e43749fb4234 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.737141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23010-0e94-4726-8f29-e43749fb4234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.758908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c7b23010-0e94-4726-8f29-e43749fb4234 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:10.761313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c7b23010-0e94-4726-8f29-e43749fb4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:10.771337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:10.773029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:10.835412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 178/346] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:10.842621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:10.843223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:10.843804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:10.876198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:10.877722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:10.889110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 169/347] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:10.890418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:10.906940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.028578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 179/348] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:11.030503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.086327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b6f05-964b-4f85-9fd1-8b507718ce5c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.104866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.157535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.158790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.160350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.161386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.162937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.165112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.167745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 170/349] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:10 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.173658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.216791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddeb2a-9874-468a-aae8-0f1b10833cde None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.262162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.262846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.263133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.263401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.264550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.266837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.268824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 180/350] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.269597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.354893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.385179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae8b29-8e89-4eef-9f09-272781a52896 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.417161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b93c1-12e8-4eb8-980a-9e7416086359 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.423875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.433403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.434288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.434861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.435573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.444084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.445933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.449395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 171/351] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.454913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.473790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.474434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.474805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.475065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.485900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:11.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:11.491742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:11.492535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 181/352] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:11.493192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.593870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db04db-6466-4a98-9f2d-c2db89d4903c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.627263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b66cc-c3e2-4814-8e00-74399a8ce17c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.632434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.633072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.633350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.633622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.634872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.636323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.638048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 172/353] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.638672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.656309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.672195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.672195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.672195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.672195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.688756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:11.693706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:11.695496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:11.697692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 182/354] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:11.701619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.740335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.791663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c904b1-4e50-41f3-8ee3-add202a66c80 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.832865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.833586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.833865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.834135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.835310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0a5be0c65ca0402189e08b37aa717cca) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:11.835620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.841581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce2f23-e917-43bd-829b-06c844ce6077 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.841715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357e739-8f15-4ec9-a19e-5627615aadfc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:11.845085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 173/355] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.846912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9daad-5164-4cc3-963b-0c2d456fcb8c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.846912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9daad-5164-4cc3-963b-0c2d456fcb8c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.848062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9daad-5164-4cc3-963b-0c2d456fcb8c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.849487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 174/356] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:11.850219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9daad-5164-4cc3-963b-0c2d456fcb8c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.891531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:11.892313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:11.892507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:11.893995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:11.894401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:11.894780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:11.896206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:11.907441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 183/357] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:11.908328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:11.954054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:11.994769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9daad-5164-4cc3-963b-0c2d456fcb8c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.013722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:12.037765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e361aa-d1e8-434c-b50d-75bff4756111 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.057236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.058230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.058946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.059253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.060096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=d7498f66178947778670d4d051df8536, scope.project.id=0a5be0c65ca0402189e08b37aa717cca) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:12.060412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:12.061857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d922c1-d5ab-4252-9a05-5b044664df90 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:12.069943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 175/358] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=d7498f66178947778670d4d051df8536 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:12.073344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d1eb-3d5f-4cfc-bd5b-57652abbfc29 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.073730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d1eb-3d5f-4cfc-bd5b-57652abbfc29 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.074076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2d1eb-3d5f-4cfc-bd5b-57652abbfc29 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.075909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 176/359] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:12.077127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826cc5-27a2-464b-82ec-f16999f346b1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.077414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826cc5-27a2-464b-82ec-f16999f346b1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.077751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826cc5-27a2-464b-82ec-f16999f346b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.086716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.088317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.088597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.088870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.089567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:12.089869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:12.091342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:12.100336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8c826cc5-27a2-464b-82ec-f16999f346b1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:12.101541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8c826cc5-27a2-464b-82ec-f16999f346b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:12.101646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 184/360] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:12.102503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:12.195422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 177/361] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:12.197568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.197975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.198336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.220615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:12.221894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:12.255099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2dfb24-a8e5-4717-a0be-2d058e953f71 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.314629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.315283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.315563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.315839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] PATH_INFO: `/v3/users/6ab7d3155489419d95e095c65db6f24a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.319125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] RBAC: Authorizing `identity:get_user(user_id=6ab7d3155489419d95e095c65db6f24a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:12.320584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:12.322439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 185/362] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:11:12 2022] GET /identity/v3/users/6ab7d3155489419d95e095c65db6f24a => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:12.323100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:12.326256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:12.335678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 178/363] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:12.336731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:12.362956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:12.458880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c096-9acf-4614-8dbb-55683bb7f85b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.498391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] RBAC: Adding query filter params (role.id=4b5bb1647c844291b8990ac8c47ba847, user.id=6ab7d3155489419d95e095c65db6f24a, scope.system=all) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:12.502600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:12.515981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 179/364] 10.222.0.20 () {56 vars in 1308 bytes} [Sun Sep 18 06:11:12 2022] GET /identity/v3/role_assignments?user.id=6ab7d3155489419d95e095c65db6f24a&scope.system=all&role.id=4b5bb1647c844291b8990ac8c47ba847 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:12.518838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c7c45-a442-46be-bef4-5052994d6079 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.573024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.573646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.573921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.574189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:12.661160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:12.662718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:12.684775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:12.769992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:12.813458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 186/365] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:12.817597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:12.872732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:12.948159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffde44d-07b6-47be-817c-a513c86c9a3f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.992264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312257e9-85da-45d4-beae-1610f55fdce6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:12.996986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:12.998168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:12.998877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:12.999489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.018108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:13.019628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:13.029630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:13.031324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:13.032170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:13.034564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 180/366] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:13.035609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.044677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.044677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.044978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.045316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.046534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.048110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.050127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 187/367] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:12 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.053754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.172819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622086-8f66-4c0d-91b5-6999b4e69573 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.192278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976acaa0-f250-41d2-8f63-d40095ebd383 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.195809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.230367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.231294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.231746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.232261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.242753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.244029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.244402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.244763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.245518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:13.245913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.246618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:13.247503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:13.252598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:13.253519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:13.254300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:13.256811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 181/368] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:13.257036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 188/369] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.261868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.263320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.314218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.375834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb2aae-36fa-415d-9df2-ece708e722ed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.388010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0ec24-8e9d-48f8-9941-115325532370 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.412456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.413023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.413298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.413561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.414226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:13.414522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.416529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.429028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 189/370] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.429028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.432667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.433563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.434053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.434521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.436047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.437805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.440078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 182/371] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.441097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.498817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.538052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a7b1f-0b8b-4213-b030-2a0783711cef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.547824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.559584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ad68c-e99b-4d4e-8bdf-d569be1936c3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.576636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.577178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.577520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.577797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.599266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.599931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.600264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.600541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.654962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.656418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.680685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.682485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.730442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.782460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 190/372] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:13.783409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.801374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:13.810848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 183/373] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:13.812040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.915480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b1d04-b177-4ec8-91c6-3fc671e01fc5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.930316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c05c-25e3-4291-8608-cbfe374c0fee None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:13.965694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.966642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.967104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.967799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.968708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=d7498f66178947778670d4d051df8536, scope.project.id=d47864572a6a4628875eb3a75e820751) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:13.969187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.970803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.972203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:13.972906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:13.973254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:13.973595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] PATH_INFO: `/v3/projects/39ab4a771cc044ca930df75a3565efaa` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:13.974997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] RBAC: Authorizing `identity:get_project(project_id=39ab4a771cc044ca930df75a3565efaa)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:13.976432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:13.978616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 184/375] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/projects/39ab4a771cc044ca930df75a3565efaa => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.979374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.981953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 191/375] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=d7498f66178947778670d4d051df8536 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:13.988603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:13.997913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:14.116871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:14.146058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a136d-954a-427b-ad2a-3cb2e9427691 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:14.155359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643ab28-d36f-474a-bc83-478fd6d009f7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:14.197179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:14.197938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.198312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.198763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.200074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:14.201638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:14.203468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 185/376] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:14.209221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:14.222918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:14.223580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.223857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.224128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] PATH_INFO: `/v3/groups/8ef5a40872764fecbfb92cc1640af282` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.238512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] RBAC: Authorizing `identity:get_group(group_id=8ef5a40872764fecbfb92cc1640af282)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:14.240050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:14.244003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 192/377] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:13 2022] GET /identity/v3/groups/8ef5a40872764fecbfb92cc1640af282 => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:14.266346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:14.358729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:14.387260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b09ca-11d9-4b20-8a07-ce9b4e37194e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:14.411354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e6ece-ee56-4792-a929-f03a89bc95cf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:14.431697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:14.432355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.433093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.433363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.434782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:14.436894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:14.438844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 186/378] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:14 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:14.445954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:14.486179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:14.487191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.487912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.489321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.490281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] RBAC: Adding query filter params (group.id=8ef5a40872764fecbfb92cc1640af282, role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=39ab4a771cc044ca930df75a3565efaa) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:14.491192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:14.496189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7b2b5-006b-47a5-aeaa-2fd350bab7ec None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:14.503332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:14.511413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 193/379] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:14 2022] GET /identity/v3/role_assignments?group.id=8ef5a40872764fecbfb92cc1640af282&scope.project.id=39ab4a771cc044ca930df75a3565efaa&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:14.612868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b7dc0-437d-4a56-a6f2-2943a8e59549 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:14.656972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:14.657586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.657870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.658141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.658907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=d47864572a6a4628875eb3a75e820751) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:14.659215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:14.660637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb6de1-9cfd-45e8-a5bb-3b7999af60cf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:14.667209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 187/380] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:14 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:14.684133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8ceb-38d2-4753-bb37-ad4d331a9402 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.684665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8ceb-38d2-4753-bb37-ad4d331a9402 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.684749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d8ceb-38d2-4753-bb37-ad4d331a9402 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.686342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 194/381] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:14.704801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddc25-6d50-4e28-b1ce-96b930f08dd0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.705130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddc25-6d50-4e28-b1ce-96b930f08dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.705350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ddc25-6d50-4e28-b1ce-96b930f08dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.733466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b26ddc25-6d50-4e28-b1ce-96b930f08dd0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:14.734746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b26ddc25-6d50-4e28-b1ce-96b930f08dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:14.738285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:14.828428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 188/382] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:14.898187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1fdf-d740-4bf0-af84-24838cefc8f3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.898187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1fdf-d740-4bf0-af84-24838cefc8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.898187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa1fdf-d740-4bf0-af84-24838cefc8f3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.899956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 195/383] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:14.906722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:14.907047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:14.907328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:14.927553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:14.928754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:15.018754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 189/384] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:15.164167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa1fdf-d740-4bf0-af84-24838cefc8f3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:15.294165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa1fdf-d740-4bf0-af84-24838cefc8f3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:15.334069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:15.334731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:15.335011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:15.335278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:15.351755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:15.356769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:15.357623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:15.358404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:15.360475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 196/385] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:15.366194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:15.472241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:15.515403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f31d6-fbe0-4474-bf7d-f014d3f57618 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:15.567208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:15.567778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:15.568058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:15.568323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:15.568986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:15.569290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:15.570679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:15.579661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 190/386] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:15.585245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:15.605797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:15.702468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:15.774750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7908e-a2a8-4193-9abb-7e37da9a3464 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:15.801343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e419f4f-be16-4b42-acce-c82f014fefa3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:15.821280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:15.822473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:15.822860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:15.823202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:15.826241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:15.828023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:15.830145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 197/387] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:15.835296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:15.874831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:15.881677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:15.882335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:15.882680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:15.882908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:15.906437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:15.911837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:15.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:15.913865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 191/388] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:15.920448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:16.009867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60473065-3d8a-4a5b-b7b9-5acaf9b4147e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.059360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:16.073017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.073585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.073864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.074138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.076197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.077891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.079554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 198/389] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:16.085094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:16.090952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be989513-2d6b-46df-b19c-3c5f023b59ed None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.132660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.133567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.134113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.134570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.135509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:16.135992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.137608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.148139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 192/390] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:16.154387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:16.207963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc1f-4cef-4ce6-a377-00ec080dd040 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.230308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:16.248837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.249893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.250416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.250986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.253421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, group_id=9eb05ec41e134a33af0153f2e1a1d359, role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.255193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.273064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 199/391] 10.222.0.20 () {58 vars in 1442 bytes} [Sun Sep 18 06:11:16 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:11:16.285184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413043d-75db-4a74-9ae1-a1fde063bd66 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.324562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.325076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.325346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.325611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.326790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.328253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.330180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 193/392] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:16 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:16.335829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:16.416515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:16.490139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50039c-6b83-4521-ba8e-eeaa81037288 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.543396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.544067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.544344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.544615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.545879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.547460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed18c1-2444-43b3-88fc-24cb8ad11f00 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.549140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 200/393] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:16 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:16.555024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:16.673448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec0434-1796-4015-b16f-f1df7a7333ee None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:16.696085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:16.709368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:16.710026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.710302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.710568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.712573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a5be0c65ca0402189e08b37aa717cca, group_id=9eb05ec41e134a33af0153f2e1a1d359, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:16.714007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9896b18-5ea1-4e3b-89b8-8a447244bf99 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:16.732266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 194/394] 10.222.0.20 () {58 vars in 1442 bytes} [Sun Sep 18 06:11:16 2022] PUT /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:16.882065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147819e0-266f-479c-9cde-102f426dfdaf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.882564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147819e0-266f-479c-9cde-102f426dfdaf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.882690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147819e0-266f-479c-9cde-102f426dfdaf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.884335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 201/395] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:16.890433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c4d9-3e92-4496-aa04-77b0ac24e2df None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:16.890934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c4d9-3e92-4496-aa04-77b0ac24e2df None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:16.891005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0c4d9-3e92-4496-aa04-77b0ac24e2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:16.915771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-07d0c4d9-3e92-4496-aa04-77b0ac24e2df None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:16.916988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-07d0c4d9-3e92-4496-aa04-77b0ac24e2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:17.000985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 195/396] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:17.377365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147819e0-266f-479c-9cde-102f426dfdaf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:17.432992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aad14-4fee-442a-89a0-b84eb701dbba None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.433337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aad14-4fee-442a-89a0-b84eb701dbba None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.433534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aad14-4fee-442a-89a0-b84eb701dbba None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.435084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 196/397] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:17.443236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c697f-3a09-4559-9013-e5378442dbd1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.443520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c697f-3a09-4559-9013-e5378442dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.445890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c697f-3a09-4559-9013-e5378442dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.469554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5d1c697f-3a09-4559-9013-e5378442dbd1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:17.470872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5d1c697f-3a09-4559-9013-e5378442dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:17.520315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147819e0-266f-479c-9cde-102f426dfdaf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:17.566979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:17.587805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:17.588449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.589801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.590081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.605366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 197/398] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:17.606723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e4d7-52a8-4c6d-938a-3cceba2123a7 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.607019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e4d7-52a8-4c6d-938a-3cceba2123a7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.607297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e4d7-52a8-4c6d-938a-3cceba2123a7 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.608799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 198/399] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:17.609036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:17.613781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.614067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.614339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:17.614505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:17.615376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:17.616564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 202/400] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:17.623617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:17.655026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:17.656215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:17.742941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:17.784327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 199/401] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:17.829082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4076c-3740-46fe-84e6-0b0fb0363f1c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:17.877845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:17.880321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:17.880937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:17.881213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:17.881482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:17.882151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:17.882714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:17.884366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:17.893353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 203/402] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:17.899588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:17.988459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8207d3-d40d-4353-bd75-35703e7a19e0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:17.994188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.025092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.025725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.026004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.026273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.036824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:18.042175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:18.043149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:18.044002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 200/403] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:18.049644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.082785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.094368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b808e-01e7-4ffc-84dc-f10f4afe35cb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.138303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.139596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.140817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.141581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.143178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.145203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.147805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 204/404] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:17 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.148425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.161384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bba5e-abaf-4c11-baa8-dcd5ba9b158e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.197431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.198025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.198300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.198566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.199252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:18.199624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.201071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.214315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 201/405] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.215196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.277435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f9b23-1ac4-46c3-9497-ed0468573927 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.291473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.312514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.313268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.313631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.314058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.317164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.324253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171709d-a8b0-454e-9850-ff61cbe2bb4d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.327071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:18.332838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:18.333712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:18.334564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 205/406] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:18.335247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.360050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.360832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.361114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.361384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.362510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.364177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.366092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 202/407] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.367139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.464630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149e79-405a-40bb-b96d-55bd4fc292c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.481879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cec01c-3e9b-46ba-a087-997597be4f60 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.509464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.510502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.510810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.511083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] PATH_INFO: `/v3/groups/8ef5a40872764fecbfb92cc1640af282` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.512437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] RBAC: Authorizing `identity:get_group(group_id=8ef5a40872764fecbfb92cc1640af282)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.514514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.521140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 206/408] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/groups/8ef5a40872764fecbfb92cc1640af282 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.521140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.524262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.525637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.526138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.526667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.527663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:18.528197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.529746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.539344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 203/409] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.540710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.569129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.570938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.650200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f81e3-f4ae-4d49-82b8-b09c6268d980 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.655328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72c543-b35f-491c-a421-49be5c2d73fa None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.685505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.686173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.686450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.686750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.689438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, group_id=9eb05ec41e134a33af0153f2e1a1d359, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.691542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.700068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.700917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.701195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.701532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] PATH_INFO: `/v3/projects/39ab4a771cc044ca930df75a3565efaa` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.702863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] RBAC: Authorizing `identity:get_project(project_id=39ab4a771cc044ca930df75a3565efaa)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.704999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.707025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 204/410] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/projects/39ab4a771cc044ca930df75a3565efaa => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.711812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.713938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 207/411] 10.222.0.20 () {58 vars in 1442 bytes} [Sun Sep 18 06:11:18 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:18.716234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.895796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.911746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a05c4-a54f-4f26-aa0c-63142428f69f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.924455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5466c-92a7-41e2-a453-01c6fc945fe6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:18.926099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:18.973313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.973998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.974274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.974542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] PATH_INFO: `/v3/projects/39ab4a771cc044ca930df75a3565efaa/groups/8ef5a40872764fecbfb92cc1640af282/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.976134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:18.976570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] RBAC: Authorizing `identity:create_grant(project_id=39ab4a771cc044ca930df75a3565efaa, group_id=8ef5a40872764fecbfb92cc1640af282, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.977220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:18.977750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:18.978071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84567f1f-a9e7-4e67-8f16-7de35dd7e4bd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.978419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:18.979832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:18.982695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:18.984602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 205/412] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:18.991015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:18.996086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 208/413] 10.222.0.20 () {58 vars in 1442 bytes} [Sun Sep 18 06:11:18 2022] PUT /identity/v3/projects/39ab4a771cc044ca930df75a3565efaa/groups/8ef5a40872764fecbfb92cc1640af282/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:19.101351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720a39-04f0-4871-b345-cd8a6179b0d9 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.101969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720a39-04f0-4871-b345-cd8a6179b0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.102457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a720a39-04f0-4871-b345-cd8a6179b0d9 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.104599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 209/414] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:19.106807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec962a1-af03-4710-89a9-955abfd4107a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:19.110384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe823-0200-4a94-a296-28096251673d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.110928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe823-0200-4a94-a296-28096251673d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.111412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe823-0200-4a94-a296-28096251673d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.145477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:19.146170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.146405 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.146705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.148920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:19.149908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:19.150919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0bdbe823-0200-4a94-a296-28096251673d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:19.151600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 206/415] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:18 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:19.152542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0bdbe823-0200-4a94-a296-28096251673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:19.158611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:19.195096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:19.284792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864f3ad-d830-43b5-b685-11dc056cd239 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:19.292416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 210/416] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:19.321404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:19.322063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.322337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.322706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.325170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] RBAC: Authorizing `identity:create_grant(project_id=d47864572a6a4628875eb3a75e820751, group_id=9eb05ec41e134a33af0153f2e1a1d359, role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:19.326636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e732cf-ba41-4a03-982b-505ae5f46623 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:19.342659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 207/417] 10.222.0.20 () {58 vars in 1442 bytes} [Sun Sep 18 06:11:19 2022] PUT /identity/v3/projects/d47864572a6a4628875eb3a75e820751/groups/9eb05ec41e134a33af0153f2e1a1d359/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:19.573197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7c0d-614e-4627-b7a4-73aaceafe822 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.573516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7c0d-614e-4627-b7a4-73aaceafe822 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.573747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a7c0d-614e-4627-b7a4-73aaceafe822 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.575429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 211/418] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:19.580997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.581224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a7c0d-614e-4627-b7a4-73aaceafe822 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:19.581318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.581548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.605987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:19.607253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:19.751042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 208/419] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:19.759689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:19.773443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a7c0d-614e-4627-b7a4-73aaceafe822 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:19.822068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:19.822777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:19.823062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:19.823335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:19.840487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:19.845324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:19.846196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:19.847028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 212/420] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:19.852268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:19.872302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:19.982112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fca71-e81d-4bf8-bf41-07314b8a046c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.024981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.025721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.025901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.026048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.026698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:20.026998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:20.028363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:20.033225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.040732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 209/421] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:20.052344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.204874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:20.218606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b99c1-1ac7-4fc1-8b76-1568b7270a44 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.246745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d42e3c-6001-4867-9914-864b70b0e20e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.286057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.287040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.287525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.288019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.297083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.297954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.299445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.299950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.308361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:20.318085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:20.319026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:20.319806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:20.322869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 213/422] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:20.331494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.347008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:20.435045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:20.438096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:20.511598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64a6a-e692-4baa-b86c-8e6a0337e813 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.548129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.549066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.549541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.550004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.550890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:20.551406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:20.553081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:20.562394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 214/423] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:20.568211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.612397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:20.662172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 210/424] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 06:11:20.667548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:20.671161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.714081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d45d1e-32bd-429b-bd1c-7b2d2499da6b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.769607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.770211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.770483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.770777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.853574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3808317-b327-4b26-af80-bc994aa435db None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:20.896541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:20.897167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:20.897439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:20.897706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:20.899135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:20.901869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:20.903561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 211/425] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:20.909070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:20.910595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:20.911506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:20.991000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:21.002953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:21.011582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 215/426] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:21.017266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:21.123983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a1398-eb18-4aee-af36-56990caac5ff None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:21.162696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:21.163685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.164104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61442a7a-7848-400a-8d64-ebc36ef22617 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:21.164751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.165760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] PATH_INFO: `/v3/projects/0a5be0c65ca0402189e08b37aa717cca` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.171364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] RBAC: Authorizing `identity:get_project(project_id=0a5be0c65ca0402189e08b37aa717cca)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:21.181453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:21.183905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 216/427] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:21 2022] GET /identity/v3/projects/0a5be0c65ca0402189e08b37aa717cca => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:21.190464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:21.245182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:21.247630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.248111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.249522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.251609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:21.254596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:21.260657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 212/428] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:20 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:21.263207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:21.283088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:21.402745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c0012-834f-4315-8e10-e3150cd82f47 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:21.436307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc1b0c-71c3-4ad7-b65b-6602856542f1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:21.443964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:21.444754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.445032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.445296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.446501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:21.448067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092bc6c-438b-4d3b-9178-17afea773dba None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:21.450037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 217/429] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:21 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:21.451541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893696c-7c30-4943-a996-89f894360b95 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.451682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893696c-7c30-4943-a996-89f894360b95 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.451864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893696c-7c30-4943-a996-89f894360b95 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.453676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 218/430] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:21.458243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:21.466081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6d56-781d-49f1-b8ca-8a957b43e41e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.466081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6d56-781d-49f1-b8ca-8a957b43e41e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.466081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf6d56-781d-49f1-b8ca-8a957b43e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.483817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2caf6d56-781d-49f1-b8ca-8a957b43e41e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:21.484018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:21.484722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.485057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.485325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.486100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0a5be0c65ca0402189e08b37aa717cca) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:21.486325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2caf6d56-781d-49f1-b8ca-8a957b43e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:21.486489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:21.488976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:21.502630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 213/431] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:21 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:21.502630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:21.627427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 219/432] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:21.660733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef055-5b20-4e7d-a5a0-5c852296eda3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.661357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef055-5b20-4e7d-a5a0-5c852296eda3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.661823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef055-5b20-4e7d-a5a0-5c852296eda3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.663724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 220/433] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:21.670348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da149-0af1-490a-9a3f-7d8a78507bad None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.671006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da149-0af1-490a-9a3f-7d8a78507bad None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.671545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104da149-0af1-490a-9a3f-7d8a78507bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.693010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-104da149-0af1-490a-9a3f-7d8a78507bad None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:21.694552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-104da149-0af1-490a-9a3f-7d8a78507bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:21.706458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088da74-4ccd-464d-a239-fb320834c768 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:21.744018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:21.752995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:21.753609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.753883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.754150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.755345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=d7498f66178947778670d4d051df8536, scope.project.id=0a5be0c65ca0402189e08b37aa717cca) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:21.755667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:21.757140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682db63d-0ba1-4146-bfab-892d70b75759 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:21.770200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 214/434] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:21 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=0a5be0c65ca0402189e08b37aa717cca&role.id=d7498f66178947778670d4d051df8536 => generated 611 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:21.793076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 221/435] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:21.895713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b2431-4a88-48e4-b8d9-8776d59e042f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.896033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b2431-4a88-48e4-b8d9-8776d59e042f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.896263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b2431-4a88-48e4-b8d9-8776d59e042f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.897800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 215/436] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:21.903457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:21.903737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:21.904011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:21.923958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:21.925104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:21.949569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b2431-4a88-48e4-b8d9-8776d59e042f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.009390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 222/437] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:22.017342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.076680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.077709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b2431-4a88-48e4-b8d9-8776d59e042f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.123842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.125378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.125378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.125378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.139486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:22.147777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:22.148671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:22.149976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 216/438] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:22.156820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.204010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.208098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5c291-dc6b-43c2-92f7-6606d14ff3b3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.259831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.260465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.260739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.261004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.272792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:22.278423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:22.279451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:22.280242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:22.280856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9a434-5551-4380-b8c1-0bdd686bd580 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.281254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 223/439] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:22.285129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.287183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.324294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.324901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.325179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.325449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.326210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:22.326520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.328792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:22.339007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 217/440] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:22.339655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.405698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6095e-88f7-479a-9e32-31e89a077907 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.445969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.447174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.447446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.447713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.454737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb56cb-1a05-4e21-b245-c1ed348c65a9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.466237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:22.471522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:22.472384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:22.473173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 224/441] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:22.473814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.494353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.494998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.495273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.495541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.496012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.496221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:22.496517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.497875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:22.506236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 218/442] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:22.507272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.534203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.576609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e57c9-9cc5-4535-9903-68380a33e353 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.604660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18439380-a65c-4534-a935-aed40370ad63 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.608860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.609459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.609734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.610001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.637470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.638057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.638335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.638628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.639290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:22.639591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.641204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:22.650069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 219/443] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:22.650878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.678283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.679809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:22.699413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.740114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.751242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d90b7-8bed-49e7-a61c-00c994460bf6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.775367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 225/444] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:22.776194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.785221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.785816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.786109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.786377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.851559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.853025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:22.873116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489e63-c3ca-4780-8d2a-839115ba568a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:22.892800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.906311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:22.906945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:22.907218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:22.907482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:22.938799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:22.950258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 220/445] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:22.951089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:22.976802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:22.978297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.044598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b1129-5696-4092-b43a-b84efcbb9db7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.072569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 226/446] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:23.073401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.077565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.078289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.078559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.078853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.080055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.081562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.083295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 221/447] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:22 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.084050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.097914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.142746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.174760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ba246-5c38-4d84-b9e6-d5bab74768cd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.185264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1be5d3-79b5-4118-9f5f-c99cda0bca2d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.208961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.209579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.209860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.210126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] PATH_INFO: `/v3/projects/39ab4a771cc044ca930df75a3565efaa` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.211250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] RBAC: Authorizing `identity:get_project(project_id=39ab4a771cc044ca930df75a3565efaa)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.212734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.214505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 227/448] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/projects/39ab4a771cc044ca930df75a3565efaa => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.215182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.224117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.224733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.225004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.225275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] PATH_INFO: `/v3/projects/d47864572a6a4628875eb3a75e820751` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.226359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] RBAC: Authorizing `identity:get_project(project_id=d47864572a6a4628875eb3a75e820751)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.228335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.229928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 222/449] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/projects/d47864572a6a4628875eb3a75e820751 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.230686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.301071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.312623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe6688-6aeb-47df-8949-843f26767619 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.328960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1082a-7dec-446a-87db-887631214307 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.345417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.346037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.346313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.346611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.347712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.349142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.350906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 228/450] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.351499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.354695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.363892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.364502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.364801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.365069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] PATH_INFO: `/v3/groups/8ef5a40872764fecbfb92cc1640af282` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.366157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] RBAC: Authorizing `identity:get_group(group_id=8ef5a40872764fecbfb92cc1640af282)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.367584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.369180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 223/451] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/groups/8ef5a40872764fecbfb92cc1640af282 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.369923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.452062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a432-bfbf-4f36-85e8-3e62f32c04fa None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.467798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e37ad-1d00-4c14-a69c-3baaf62ab717 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.488239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.488863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.489135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.489400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] PATH_INFO: `/v3/groups/9eb05ec41e134a33af0153f2e1a1d359` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.490482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] RBAC: Authorizing `identity:get_group(group_id=9eb05ec41e134a33af0153f2e1a1d359)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.491974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.494005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 229/452] 10.222.0.20 () {56 vars in 1180 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/groups/9eb05ec41e134a33af0153f2e1a1d359 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.494773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.504094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.505042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.505328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.505600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.506682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=d7498f66178947778670d4d051df8536, scope.project.id=d47864572a6a4628875eb3a75e820751) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:23.507145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.508557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.514800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 224/453] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=d7498f66178947778670d4d051df8536 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.515977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:23.519414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.595722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:23.603913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5ae8-3e0b-41c1-9870-0e608307f50d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.642428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.642776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134d2bb-f7c6-4974-bb3c-b04add8f068a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:23.643188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.643479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.643741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.644489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] RBAC: Adding query filter params (group.id=8ef5a40872764fecbfb92cc1640af282, role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=39ab4a771cc044ca930df75a3565efaa) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:23.644796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.646279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd1d18-4ba4-423f-9965-ee00cfa6c142 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.659113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 230/454] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/role_assignments?group.id=8ef5a40872764fecbfb92cc1640af282&scope.project.id=39ab4a771cc044ca930df75a3565efaa&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:23.682983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:23.683608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:23.683886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:23.684155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:23.684904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] RBAC: Adding query filter params (group.id=9eb05ec41e134a33af0153f2e1a1d359, role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=d47864572a6a4628875eb3a75e820751) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:23.685211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:23.686688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d080b-ea61-40a5-bfd2-cef7b7fdfd57 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:23.694805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 225/455] 10.222.0.20 () {56 vars in 1376 bytes} [Sun Sep 18 06:11:23 2022] GET /identity/v3/role_assignments?group.id=9eb05ec41e134a33af0153f2e1a1d359&scope.project.id=d47864572a6a4628875eb3a75e820751&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:26.441796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a100c0a-a717-4035-97f2-a934bab641c2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.442310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a100c0a-a717-4035-97f2-a934bab641c2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.442310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a100c0a-a717-4035-97f2-a934bab641c2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.443959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 231/456] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:26.449111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926200-0bf3-4ba5-b183-f145a9681082 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.449891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926200-0bf3-4ba5-b183-f145a9681082 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.450354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926200-0bf3-4ba5-b183-f145a9681082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.479551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-20926200-0bf3-4ba5-b183-f145a9681082 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:26.481057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-20926200-0bf3-4ba5-b183-f145a9681082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:26.569990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 226/457] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:26.771174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463146ed-6d26-4186-b4fc-46152865b30c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.771367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463146ed-6d26-4186-b4fc-46152865b30c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.771813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463146ed-6d26-4186-b4fc-46152865b30c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.773525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 232/458] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:26.779694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.779981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.780249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.780834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebc3cc-c552-49a3-bbf1-e159705f7d36 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.781121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebc3cc-c552-49a3-bbf1-e159705f7d36 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.781392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebc3cc-c552-49a3-bbf1-e159705f7d36 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.782746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 233/459] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:26.787664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:26.788148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:26.788636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:26.807570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:26.809318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:26.811807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:26.813806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:26.886536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:26.903777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 234/460] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:26.913190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 227/461] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:26.993271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.021690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.127489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eacee4-39f8-44ec-bb5f-63d1d251f2eb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.166986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.167623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.167901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.168174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.184153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.185780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:27.213264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:27.214237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:27.215124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:27.215124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.215124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:27.215294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 235/462] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:27.223189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.264829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.349960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.363511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd827339-22e0-4612-9ee1-c55487946df8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.383569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa00af-cf83-45c8-b95f-8bce04f4c274 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.401583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.402195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.402474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.402811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.403446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:27.403745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.405108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790cd43b-0cc5-4f86-8b3a-d6b91aed62f4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.418696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 228/463] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:27.418696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14974f28-a006-4a86-a0a7-fcbed9c242a4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.419111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14974f28-a006-4a86-a0a7-fcbed9c242a4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.419245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14974f28-a006-4a86-a0a7-fcbed9c242a4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.420549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 229/464] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:27.421117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14974f28-a006-4a86-a0a7-fcbed9c242a4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.423418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.424209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.424621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.425035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.436359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.437810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:27.452844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:27.454154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:27.455556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:27.455556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.455556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:27.455556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 236/465] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:27.455556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.482041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.548893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14974f28-a006-4a86-a0a7-fcbed9c242a4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.569504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db127576-2e30-48ce-a938-71e663bba980 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.589036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.589662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.589941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.590214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.596978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.602216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.604263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.610295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.611334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.611614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.611880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.612910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.614479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-15c388f9-f337-4e7e-82a3-809981e09afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:27.618505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:27.619475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:27.620293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:27.620293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:27.620293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:27.620293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 230/466] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:27.621098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.621379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.621650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.639932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 237/467] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:27 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:27.641477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:27.641683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.642847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:27.739117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 231/468] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:27.740984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.768443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd616d-19e1-4f08-9a1f-6873872d6dfe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.799239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.841965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.842594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.842881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.843146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.843795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:27.844094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.845703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.869797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 238/469] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:27.876277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:27.918953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b8823-8ae9-401e-a71e-468b04e1bcd4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:27.933892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:27.975692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:27.976316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:27.976590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:27.976853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:27.977543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:27.977838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:27.979370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:27.995113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 232/470] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:28.002400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:28.015247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b83243-4e69-4370-a6a8-6542af0eb833 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.057621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.058235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.058510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.058812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.059462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.061440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.089939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 239/471] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:27 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:28.090867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:28.131088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:28.142568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51f0bb-302a-43f2-a6cc-e071d3775c9b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.187215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.187860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.188144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.188432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.189452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.192571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb7759-1ebb-413a-bf30-90327557baa1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.202971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:28.211444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d54c9-3d24-420e-aae8-f4407c4e3207 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.224341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 233/472] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:28 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:11:28.226480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ee122-620c-4722-b1c9-3a2d7092fb08 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.226792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ee122-620c-4722-b1c9-3a2d7092fb08 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.227076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ee122-620c-4722-b1c9-3a2d7092fb08 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.233497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 234/473] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:28.235681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.235976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.236249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.258331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:28.260144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:28.289172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.289921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.290200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.290552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.303959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.308035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:28.328568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:28.329547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:28.330397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:28.330397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:28.330397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.330397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 240/474] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:28.336525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:28.349690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 235/475] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:28.465022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44bbbef-373d-4a98-95c9-2648bfe45faf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.502843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.503453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.503728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.503997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.504643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:28.504942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.506336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.519957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 241/476] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:28.526000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:28.601283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:28.635616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2161a0e-1c42-4888-bcf1-014fe960e14e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.659640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:28.670378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:28.677636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.678252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.678711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.679025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.679686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.681157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.705749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 236/477] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:28 2022] POST /identity/v3/users => generated 249 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:11:28.818215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6592ef-29cc-4bb5-9c3d-72ab9246f295 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:28.872610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:28.873239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:28.873518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:28.873786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:28.874529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:28.876044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:28.890406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.sql.core [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 06:11:28.890406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 18 06:11:28.890406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [parameters: {'id': 'a49abd11f00e4fdb8a13a481f7d7249e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 18 06:11:28.890406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81397) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Sep 18 06:11:28.907051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return ref.to_dict() Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:11:28.907955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application yield resource Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise self.value Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application session.commit() Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 06:11:28.908848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.flush() Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application compat.raise_( Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 06:11:28.909765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:11:28.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:11:28.911493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [parameters: {'id': 'a49abd11f00e4fdb8a13a481f7d7249e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:28.912279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 18 06:11:28.914592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:28.916734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 242/478] 10.222.0.20 () {60 vars in 1131 bytes} [Sun Sep 18 06:11:28 2022] POST /identity/v3/roles => generated 144 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:28.916872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:29.030391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a654ace-d22a-48fa-81c1-1259dedba4de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:29.057353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:29.067217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:29.068175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:29.069460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:29.069969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:29.081750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a8f05896-130b-4db6-99e2-5a926a424c75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:29.087780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:29.088649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 18 06:11:29.089430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:29.091563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 237/479] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:11:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:29.097849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:29.176407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:29.214292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c3e2c-ffa9-4ea7-97a3-418eef4c7897 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:29.252015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:29.252996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:29.253457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:29.253904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:29.254795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:29.255300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:29.256933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790fcff-3f0b-4ca8-b1f4-4b96292d9d70 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:29.267056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 243/480] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:30.471406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345850e-de26-4a39-94e8-da97c9b85469 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:30.472241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345850e-de26-4a39-94e8-da97c9b85469 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:30.472717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345850e-de26-4a39-94e8-da97c9b85469 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:30.474504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 238/481] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:30.479482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49560-9f2c-444f-8d30-07de57333e94 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:30.480283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49560-9f2c-444f-8d30-07de57333e94 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:30.480781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f49560-9f2c-444f-8d30-07de57333e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:30.506639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a1f49560-9f2c-444f-8d30-07de57333e94 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:30.508217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a1f49560-9f2c-444f-8d30-07de57333e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:30.591970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 244/482] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:30.781511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4e8a-1dd2-4e14-b108-9f9ca885255b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:30.781709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4e8a-1dd2-4e14-b108-9f9ca885255b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:30.781970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4e8a-1dd2-4e14-b108-9f9ca885255b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:30.783559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 239/483] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:30.788742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431b-a099-4867-a193-d8b224014645 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:30.789090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431b-a099-4867-a193-d8b224014645 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:30.789374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279431b-a099-4867-a193-d8b224014645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:30.809802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7279431b-a099-4867-a193-d8b224014645 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:30.811057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7279431b-a099-4867-a193-d8b224014645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:30.891677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:30.908063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 245/484] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:31.037830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4e8a-1dd2-4e14-b108-9f9ca885255b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.070452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fc96e-e3e6-48b6-99b5-7d94f440206a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.070891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fc96e-e3e6-48b6-99b5-7d94f440206a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.071207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fc96e-e3e6-48b6-99b5-7d94f440206a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.072788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 246/485] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:31.078024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.078323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.078568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.120553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:31.121743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:31.151002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4e8a-1dd2-4e14-b108-9f9ca885255b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.187432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.188384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.188856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.189315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.200335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:31.205419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:31.206254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.207092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:31.209722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 240/486] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:31.216169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.264721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 247/487] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:31.267818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:31.276105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.353861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0183f9-c1bd-492b-b77a-715b924612b7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.393605 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.395101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.395577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.396062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.396932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:31.397433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:31.399432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:31.410344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 241/488] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:31.417346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.534402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d50631-0dd9-4fe5-be70-6c07e842b527 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.546383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06c622a-f0f0-4816-aa9f-390812dae63d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.557560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:31.562708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:31.576737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.577375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.577654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.577925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.586475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.587289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.587701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.588103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.588344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-638af06d-9c66-4707-a486-b18947a7a99d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:31.593681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:31.594541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:31.595605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 248/489] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:31.599490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6e253-9751-4c50-a7f9-eff92c63750f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.599490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6e253-9751-4c50-a7f9-eff92c63750f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.599490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6e253-9751-4c50-a7f9-eff92c63750f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.599739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:31.599816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 249/490] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:31.600022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6e253-9751-4c50-a7f9-eff92c63750f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.601485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:31.616568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:31.617617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:31.619277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:31.619277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:31.619277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:31.619776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 242/491] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:31.620640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.712924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6e253-9751-4c50-a7f9-eff92c63750f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.725696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4270-f82c-4908-aebc-071480f20773 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.749114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.749698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.749975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.750250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.760762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:31.761812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.762411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.762714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.762990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.763655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:31.763955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:31.765582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7cf379e9-027b-47ff-99a8-fde8c6892f28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:31.765837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:31.766939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:31.767738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 250/492] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:31.768743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.768951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.769226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.779143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 243/493] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:31.779823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.793411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:31.794658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:31.804916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:31.833099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:31.869008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 251/494] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:31.869914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.894844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731221-7bd4-45e8-b069-95584814b400 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:31.937686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:31.939123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:31.939754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:31.940349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:31.941248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:31.941748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:31.943402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:31.969441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 244/495] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:31.970628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:31.985468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d301fa8-a424-412b-8411-aa7b5060390c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.021709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.022323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.022617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.022890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.023729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:32.024095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.025467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.034063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 252/496] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:32.034911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.060587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.069656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.082248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853db8c6-e2b4-4d5c-8c5b-218d01e3f48a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.120573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.121569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.122094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.122634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.134207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.135995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.140477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6a9e2-910b-417f-beb0-8dcb90b5ac4f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:32.152117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.153175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:32.154002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:32.154002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:32.154002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:32.154002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 245/497] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.155416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.177303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.177935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.178212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.178483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.186469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.188220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:32.203076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.204003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:32.204879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 253/498] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.206015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.274531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611c066-bd4b-40ee-ab25-f364331b3174 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.304032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.312163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.312313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4362bcb-3a93-4c16-b771-df9f2e9f5a1a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.312783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.313061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.313329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.315569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.325325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.326801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:32.341646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.342572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:32.343483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:32.343483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:32.343483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:32.343483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 246/499] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.343694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.353418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.354059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.354341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.354634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.364763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:32.370119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.371153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:32.371973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 254/500] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.372620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.447831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612acb85-8c18-4fd3-b7be-3801be141b44 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.469931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cac4ba-c491-4715-aadc-61b27f7de47b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.483421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.484013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.484284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.484548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.485174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:32.485472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.486848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.499034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 247/501] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:32.503709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.507626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.508229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.509102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.509374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.510106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:32.510410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.511893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ea097-245f-4b77-b10a-fd1a71ab154b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.545940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.547534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 255/502] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:32.548966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5490-94f0-4dd4-be92-d7f10fff3972 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.549255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5490-94f0-4dd4-be92-d7f10fff3972 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.549535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5490-94f0-4dd4-be92-d7f10fff3972 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.550920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 256/503] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:32.551479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5490-94f0-4dd4-be92-d7f10fff3972 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.565916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.603409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae49c1-2b8b-4154-ab76-41f7e359821f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.637946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.638470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.638778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.639044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.639671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:32.640255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.641575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.651156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5490-94f0-4dd4-be92-d7f10fff3972 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.652760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 248/504] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:32.653533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.684939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.685539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.685819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.686092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.686788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:32.687092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.688480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157d853-f97f-49ca-9ac6-d5cc0e42bbc4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.696252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 257/505] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:32.696252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.696252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.696252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.715718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:32.716912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:32.759344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e6226-a09b-41a8-9dbe-7d0f8caeefe9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.766271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.781000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 258/506] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:32.781809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.793127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.793749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.794027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.794296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.798793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:32.801917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.803435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:32.816532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.817447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:32.818326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:32.822298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 249/507] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.823025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:32.901742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f38ec-cac1-4f12-bb3a-dc2f06f0aba9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.931814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0157e4b3-6ccd-4070-8f71-f241b98411f6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:32.937261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.937866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.938138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.938502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.966463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:32.967429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:32.967848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:32.968126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:32.975679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:32.977648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:32.988833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:32.989726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:32.990640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 250/508] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:32.991217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.012240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.013810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.016792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.022096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.094617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7c4e0-ce59-4d59-ba41-2540d2ceae97 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.113040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 259/509] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:33.113040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.129275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.129887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.130161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.130426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.141671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.143178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:33.160002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:33.160984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:33.161799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:33.161799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.161799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:33.161799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 251/510] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:33.161799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.212010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bc387-f0dd-4512-980d-3ca1332baa5e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.214214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.242738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.246129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.246842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.247052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.247323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.248062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:33.248358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.249798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.263961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062ad8-0cfe-48ec-b1ff-55df4e6e6da7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.266878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 260/511] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:33.271547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.299946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.300536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.300816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.301089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.301807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:33.302105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.303555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.339235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 252/512] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:33.340043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.367964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c7239-6eb3-4844-9907-91e80e3ac0d2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.401407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.402041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.402319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.402862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.412598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.413999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:33.428680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:33.429601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:33.430469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:33.430469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.430469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:33.430469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 261/513] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:33.430710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.441340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58496427-1960-4b9b-9f50-56f706f4cc8d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.446317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.480258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.480878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.481153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.481418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.484024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.492323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.493940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:33.508196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:33.509296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:33.510184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:33.510184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:33.510184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:33.510184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 253/514] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:33.510393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.533471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de403ff-2fc6-4196-9eaf-51ad3d385691 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.566037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.566745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.567000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.567275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.567915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:33.568227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.569607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.580886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 262/515] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:33.581668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.608013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a942daa-aff4-4701-a4e0-fe738db1ce33 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.641134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.641727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.642002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.642272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.654571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.679420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499fe45-aa45-46ae-ac5c-9d42d49552a5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.697276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.709949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.712040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.712151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.712690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.712962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.713224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.778252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.779755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.805785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 254/516] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:33.806620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.850338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.876490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 263/517] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:33.877301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.893316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:33.905177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588a268-29e7-461c-a59a-24047fd6a920 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:33.938705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:33.939236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:33.939512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:33.939785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:33.940419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:33.940715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:33.942089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:33.953264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 255/518] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:33.954035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:33.976626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb664640-cad5-4da9-901d-cb9699b089fd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.010294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.010926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.011212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.011482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.012117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:34.012420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.013790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.028981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 264/519] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:34.028981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.055085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17114877-9be3-494c-a751-62504fc3ee8f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.058083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.089331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.090017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.090293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.090560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.098838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.100290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:34.111383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:34.112240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:34.113093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 256/520] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:34.113699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.124271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a7cb1-ecfc-4b23-8c78-21d846364ece None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.128210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.160772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.161685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.161962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.162231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.176114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.177589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:34.192960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:34.194054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:34.194980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:34.194980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:34.194980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:34.194980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 265/521] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:34.195347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.218929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341021ea-d051-4ec4-9ec5-6b666ba7ada9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.252107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.252716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.252991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.253257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.264534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.266254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.277350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:34.280669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:34.281742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:34.282770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:34.282770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:34.282770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:34.282770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 257/522] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:34.283061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.296808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adde1a-daf3-46b7-a88a-cb5ef375d2d8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.328935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.329544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.329825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.330097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.330776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.332257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.340290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.353446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 266/523] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:34 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:34.354254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.385139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a1ffd-119b-4c18-8ed3-19fc8f532b92 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.423743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.424379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.424654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.424924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.434307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.436131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:34.449790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:34.450700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:34.451566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:34.455853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 258/524] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:34.456739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.500537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753bc86-fa0a-48b1-8360-0f1b86a85c2f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.538646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.539321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.540110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.540381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.541171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:34.541410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.543491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.558356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.563486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 267/525] 10.222.0.20 () {56 vars in 1139 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:34.564619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.577038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb717e-9fbf-4568-99c5-c155b2e89352 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.598538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.614191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.614863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.615085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.615357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.616054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:34.616355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.617894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.631092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 259/526] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:34.631907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.670632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b4b08-c391-424b-863d-9c0665296134 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.706363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.706917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.707195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.707462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.708101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:34.708519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.709883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.722759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 268/527] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:34.722759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.738264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09196224-68c6-401d-b767-bdbf3f12488d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.776678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.777853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.777853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.778615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.779091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:34.779394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.780811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.788714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.799278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 260/528] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:34.800086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:34.842320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:34.844830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c657ce-7bf2-4f8e-a056-6fafded2678e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.882284 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.882953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.883236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.883508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.908441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce90232-b205-462e-a3dc-26c2af944294 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:34.945241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:34.946278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:34.946783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:34.947151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:34.952632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.954096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.954608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:34.956098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:34.969185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:34.970061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:34.970940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 261/529] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:34.971409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.018797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.065400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 269/530] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:35.069861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.076997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.080718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5ffbd-462b-4a26-a7fd-26970fbd3953 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.118108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.118774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.119050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.119316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.126801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.128251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:35.140496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:35.141324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:35.142158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 262/531] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:35.142657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.176627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6e72a-ce46-4864-8e83-083c298488b4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.213539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.214150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.214440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.214731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.215462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.215768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.217225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.222878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 270/532] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:35.223498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.252605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.280611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d770d1-8992-4d78-9b39-46050d9f772f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.347280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.348552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.349804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.350456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.352276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.352996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.356495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.370115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ed279-fd81-437d-a687-4ca9f0b25d74 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.381870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 263/533] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:35.384241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.424886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.431237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.432528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.433177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.433849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.452860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.455925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:35.486513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:35.488519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:35.490532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:35.490532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:35.490532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:35.499174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 271/534] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:35.500064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.550951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4139db-dba5-42af-b703-2b19722fbc71 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.590951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.591190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.591576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.591877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.592159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.594007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.594320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.595910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.614329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 264/535] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:35.615657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.625153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad9bf6-06b7-4047-a81f-e47522a55cfd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.663419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.664094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.664380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.664815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.665592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=db771368757c449887be80383cceedbb) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.665900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.667469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.674035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 272/536] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/role_assignments?user.id=db771368757c449887be80383cceedbb&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:35.675029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.709991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:35.756692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a012c7-c900-45db-8773-7d0452b48f3e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.838805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.839427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.839705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.843316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.844025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.844380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.848102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.868155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 265/537] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:35.873702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd60c2-cb58-404f-af8d-564d7e07231e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:35.880157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:35.917062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:35.918076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:35.918551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:35.919049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:35.927428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=2e5439bf8f1f480faef63316021d4b27) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:35.928000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:35.929601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:35.944868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 273/538] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/role_assignments?user.id=2e5439bf8f1f480faef63316021d4b27&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:36.026968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfb8b-8374-4664-a356-cf216858ed08 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:36.065370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:36.066008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:36.066288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:36.066555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:36.075025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:36.076516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:36.079747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:36.090871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:36.091801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:36.092635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 266/539] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:36.098994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:36.114982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:36.214054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5fcdc-5034-4af4-9b5b-1800f613e506 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:36.253720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:36.254715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:36.255193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:36.255658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:36.256540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:36.257055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:36.259373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73653c-7b71-4dc3-9b55-77066da2f456 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:36.286132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 274/540] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:36.291779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:36.486171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd9c00-7f2e-48c7-960d-5a031b9ea083 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:36.578930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:36.579453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:36.579727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:36.579997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:36.580720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=8e742daf773049bf814cde4544b8a673) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:36.581024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:36.582447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aa012-744c-45ff-a82c-ff368d8bad4a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:36.603314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 267/541] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:36 2022] GET /identity/v3/role_assignments?user.id=8e742daf773049bf814cde4544b8a673&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:37.009500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a930-b168-46f5-a8fc-bc84779f7bfa None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:37.010035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a930-b168-46f5-a8fc-bc84779f7bfa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:37.010035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5a930-b168-46f5-a8fc-bc84779f7bfa None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:37.011404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 275/542] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:37.018071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:37.018469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:37.018658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:37.043533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:37.044750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:37.108623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:37.121215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 268/543] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:37.484743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a930-b168-46f5-a8fc-bc84779f7bfa None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:37.602776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:37.677924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5a930-b168-46f5-a8fc-bc84779f7bfa None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:37.755098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:37.756088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:37.756584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:37.757131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:37.783393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:37.793546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:37.794420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:37.795243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:37.795841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 276/544] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:37.801917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:37.942860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978dd956-31c9-4d8f-a0e6-49348efa28d0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:37.988571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:37.989521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:37.990014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:37.990507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:37.991432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:37.991920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:37.993996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e85c94-9f06-4704-bf59-3beacb52c963 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:38.006249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 269/545] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:38.012793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.099549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a241f3-ae81-435a-af35-5f94e3704938 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.100569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a241f3-ae81-435a-af35-5f94e3704938 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.101033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a241f3-ae81-435a-af35-5f94e3704938 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.105292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 270/546] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:38.112284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.112874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.113338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.131615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:38.133931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:38.210011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523ac3d-9cfc-4bfb-87d7-e512273d7cd8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:38.225589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:38.250429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 271/547] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:38.266020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:38.279219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:38.279919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.280204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.280476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.297653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:38.303917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:38.322883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:38.323757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:38.324593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:38.324593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:38.324593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:38.324593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 277/548] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:38.329695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.454684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4dab3-6dd6-45e1-ae11-2637bf2043fe None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:38.503912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:38.504526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.504805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.505074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.505730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:38.506025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:38.507445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:38.524484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 272/549] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:38.528180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.557407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.639230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:38.682275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb158b49-e0b8-45d5-becb-7cb76a3f99d3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:38.707673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee0ae1-460e-4474-a62d-fce8c729cf33 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:38.727831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:38.728715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.729018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.729642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.738968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:38.740563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:38.748609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:38.749237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.749518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.749791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-56e7f74f-1253-4bcf-8e85-e668184db97d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:38.752944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:38.753955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:38.754875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:38.759427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 278/550] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:38.760821 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a101-72ae-4864-bea6-2127edea68c3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.761111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a101-72ae-4864-bea6-2127edea68c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.761656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a101-72ae-4864-bea6-2127edea68c3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.762010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:38.763103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 279/551] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:38.764341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3414046-6f8c-4016-a04f-10d5198797bd None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.764341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3414046-6f8c-4016-a04f-10d5198797bd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.764529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3414046-6f8c-4016-a04f-10d5198797bd None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.766175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 280/552] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:38.766913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3414046-6f8c-4016-a04f-10d5198797bd None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fabd7e25-9bda-4240-8b5d-03cf1a7b3363 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:38.771692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:38.772676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:38.773906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 273/553] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:38.774716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a423-f3a0-4cdc-b367-ae44623f7b2d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.774716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a423-f3a0-4cdc-b367-ae44623f7b2d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.774849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a423-f3a0-4cdc-b367-ae44623f7b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.793260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fe29a423-f3a0-4cdc-b367-ae44623f7b2d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:38.794452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fe29a423-f3a0-4cdc-b367-ae44623f7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:38.807744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:38.870784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 274/554] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:38.872307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.872593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.872859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.892772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:38.894023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:38.900848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3414046-6f8c-4016-a04f-10d5198797bd None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:38.925216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:38.945733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:38.946215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:38.946493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:38.946801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:38.947617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:38.947986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:38.949397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:38.971437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 281/555] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:38.972304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:38.974868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 275/556] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:38.977526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.119889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee49779-db9e-4c46-b7d3-9e3eacccba57 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.174302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.178474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.179339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.179812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.180425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.182335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a1324-a2ed-4ba0-9e11-47dfe96595b7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.220621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.222023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.223632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.223920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.224250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.224968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:39.225266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.226633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.238662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 282/557] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:39.239836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.266268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.267901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.346249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd6f18-c89d-4ecd-8e5c-72eb6e82b7bf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.378218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 276/558] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:39.378218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.384411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.385050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.385324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.385592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.396148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:39.401884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:39.402800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:39.403694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 283/559] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:39.404793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.423459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.448037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.484074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c32cbb-7916-4560-8c77-1c0423e84c77 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.522035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485c459-e18d-4563-a1c4-84e3c209bf77 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.525383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.526032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.526307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.526571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.538318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.540265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.558398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:39.558695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:39.559604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:39.565304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:39.565304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:39.565304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:39.565481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.565481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.565481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.565672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 277/560] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:39.565672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.576322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:39.581100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:39.581977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:39.582849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 284/561] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:39.584366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.664269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.671228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57f265-d57b-4f8d-b057-8c3d107ed723 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.686781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d71d93-84b5-4974-90aa-1faec7d11512 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.688049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.705859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.706472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.706788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.707058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.718092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.719549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.720584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.721305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.721583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.721910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.722570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:39.722901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.724260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.732026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 285/562] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:39.732825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:39.734781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:39.735665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:39.739736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:39.739736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:39.739736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:39.739736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 278/563] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:39.739736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.831266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15015f-c724-4ff6-8796-ff95c27c5d25 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.842967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bca40-ec0b-47af-80c3-2b757df5cbe8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.863389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.863979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.864255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.864520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.865142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:39.865440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.866835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7968a-87d2-441d-b659-433b7b593cd9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.873515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.877140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:39.877725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.877996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.878258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.878794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 286/564] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:39.878987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:39.879240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:39.879972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed879-7180-46c8-bf7a-3de85554c4b4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:39.880253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed879-7180-46c8-bf7a-3de85554c4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:39.880526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ed879-7180-46c8-bf7a-3de85554c4b4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:39.880679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:39.886333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 287/565] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:39.887124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ed879-7180-46c8-bf7a-3de85554c4b4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.894171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 279/566] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:39.894171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:39.922663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:39.984859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ed879-7180-46c8-bf7a-3de85554c4b4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:39.991742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619f5e1-2990-4925-9f2f-86c580647d65 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.024380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.025017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.025291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.025559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.027345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.028316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.028596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.028877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.029501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:40.029797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.031232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.036044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.037489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.045376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 280/567] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:40.046193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f51376b6-e15c-4fbb-96a4-cf2596de77d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:40.056463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.057428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:40.058314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:40.058314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.058314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:40.058314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 288/568] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.059025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.059366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.059642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.076733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:40.078004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:40.126523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.138358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.147564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 289/569] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:40.147993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb4215-13ed-40b1-97a3-9fbbb753a258 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.148372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.186648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.187278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.187553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.187823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.196871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.198304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:40.210211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.211767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 281/570] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.212711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.255382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680108c8-1b43-4de8-8880-49c5ab862a7e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.290019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.290690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.291017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.291295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.301288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.302773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:40.318107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.322061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:40.323045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:40.323045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.323045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:40.323045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 290/571] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.323045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.332302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62484812-e334-4927-abff-f166e7e8e4b4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.361092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.369102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.369746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.370019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.370284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.377185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.379276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:40.395897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.397038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 282/572] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.397917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.417465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.442713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b78c7e-6f25-42fe-8a95-3b58149884b3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.479168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.479767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.480044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.480310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.480940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:40.481235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.482621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.494265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 291/573] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:40.495071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.503445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c7991-e0cd-4c57-9070-906a73cc625a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.537050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.537642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.537918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.538187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.538934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:40.539767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.541184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.558314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 283/574] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:40.559907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.583995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.592831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0622f-e452-4079-981e-a65c303922e8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.626282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.626497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.627137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.627417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.627692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.628736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:40.629090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.630468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.645955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 292/575] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:40.645955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.660812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140c334-4b29-4bc2-bd25-6ba98b365bf7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.695997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.696514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.696796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.697065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.697780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:40.698083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.699546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.710886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 284/576] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:40.712195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.743206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d175ab4-5625-4407-814d-65ac0840f80d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.776319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.776947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.777224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.777491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.787183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:40.792481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.793625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:40.794454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 293/577] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.795172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.810291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.813038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789674-7c7d-46f2-b27c-1a939bb8e55c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.840687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:40.846684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.847302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.847574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.847839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.855875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.857314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:40.868482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.869394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:40.870315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 285/578] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.870918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.895368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96e61b-f319-412b-921e-70ec307f798c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:40.927918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:40.928536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:40.928814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:40.929079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:40.939868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:40.941314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:40.955325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:40.956266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:40.957144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:40.957144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:40.957144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:40.957144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 294/579] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:40.957144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:40.970265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dee4d-51f0-479d-a22b-28f845e7dbad None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.004316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.004927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.005205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.005475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.012911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.014328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.015391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:41.027068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:41.028164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 286/580] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:41.029086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.057373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7acc7a-c042-479a-9f72-1241fc3f9d47 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.076916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.090662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.091263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.091538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.091863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.092593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=d763c798dc824e92b5e51deb0e059ea9) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.092881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.094295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.099885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 295/581] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:40 2022] GET /identity/v3/role_assignments?user.id=d763c798dc824e92b5e51deb0e059ea9&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.100651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.132664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736a2f4-f20e-4b65-a251-df5a1ccaabc0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.171749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.172428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.172638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.172906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.173587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.173888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.175535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.184968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 287/582] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.185849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.213133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2cbb0-d1d2-41fe-908a-c6f6edabe546 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.242660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.252860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.253543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.253847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.254200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.254979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.255291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.256887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.278015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 296/583] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.279146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.300720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab355ac2-890b-46bb-93b6-2deb722acc3d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.315096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.340690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.341622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.341895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.342200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.342825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.343136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.344711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.362959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 288/584] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.362959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.386460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26bd01-3a1b-4298-bedc-a124e54ce877 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.420465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.421074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.421351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.421616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.422343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.422669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.424087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.446357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 297/585] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.447273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.475981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eefe5-ca9a-4f89-b368-259f0e496e09 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.499772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.512674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.513297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.513578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.513846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.524785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.526274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:41.544328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:41.545303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:41.546939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:41.546939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.546939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:41.546939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 289/586] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:41.547174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.573788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.576452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76faf9cf-3df6-41d0-92a6-1ab05f33fb11 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.612862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.613750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.614212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.614649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.625625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.627129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:41.642488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:41.643698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:41.644625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:41.644625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.644625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:41.644625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 298/587] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:41.644869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.659139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220a1ec-fb9d-4d9c-9f5e-2f5f4e12a91e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.692317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.693160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.693528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.693889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.700486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.702001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:41.713911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:41.714972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 290/588] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:41.715942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.756859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.766218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6300ee8-09cf-44c5-a474-58d15ab2bc0a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.802495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.803440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.803716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.803820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:41.803982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.814682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.816136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.823610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc55c0-9166-4dec-bc88-0ef117db3623 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:41.830068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:41.831024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:41.831887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:41.831887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:41.831887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:41.831887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 299/589] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:41.832117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.865267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.865872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.866149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.866420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.867251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.868645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.870023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.882296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 291/590] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.884003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.931722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3bb43-f899-4189-89d2-5961ad9568d3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:41.972065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:41.972600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:41.972886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:41.973157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:41.973810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:41.974116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:41.975600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:41.988898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 300/591] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:41.989748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:41.999736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638931-7786-46ae-84b6-8e59ee263367 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.003330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.037845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13279-8019-432c-8827-31282a001555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.038827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13279-8019-432c-8827-31282a001555 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.039108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13279-8019-432c-8827-31282a001555 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.039937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13279-8019-432c-8827-31282a001555 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.041087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a13279-8019-432c-8827-31282a001555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:42.041681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a13279-8019-432c-8827-31282a001555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.043139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a13279-8019-432c-8827-31282a001555 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.060494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 292/592] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:42.060494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13279-8019-432c-8827-31282a001555 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.064566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.097976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18b7e1-8e39-42dd-a181-0f6f8442e3a3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.133925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.134536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.134840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.135117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.135754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:42.136055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.137427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.153746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 301/593] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:42.153746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.179083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13279-8019-432c-8827-31282a001555 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.214918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.215543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.215822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.216085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.224230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.225602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.226263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:42.236805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:42.237773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:42.238773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 293/594] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:42.239364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.267464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144cfdc-f85e-47e3-8556-beb26072623b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.302329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.303148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.303521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.303891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.311769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.313334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.314010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:42.329526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:42.330529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:42.331651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:42.332933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 302/595] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:42.333815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.357079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45753e-f9b9-44cf-ae4f-16d2b438b14c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.391765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.392440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.392717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.392992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.407507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=04ddfa59d1834ba3a7f1f27e963b090e, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.408927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.422906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 294/596] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:42 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:42.423701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.449745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5098ba-fdae-4f3b-88e5-c6af881ef469 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.493973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.506917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.507926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.509286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.509715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.519736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.521696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:42.540079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:42.541251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:42.542447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:42.543971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 303/597] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:42.544946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.566163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dba728-5b98-42d9-92f3-400fe8e02f89 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.588457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.621600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.622794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.623220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.623635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.625655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:42.626128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.627724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.651260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 295/598] 10.222.0.20 () {56 vars in 1139 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:42.652637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.664490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc6b91-e8e0-4aac-9198-9bf25ae3ec03 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.703458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.704106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.704549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.704666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.705350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:42.705691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.707299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.720744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 304/599] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:42.721593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.770697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53103a7-dbc0-4e5e-a02d-da97070288e3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.782963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.813024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.813582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.813869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.814141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.814846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:42.815168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.816576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.829548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 296/600] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:42.830415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.847043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e158a8b-fe7d-4fdc-85ff-e75a8fae2571 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:42.864509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:42.889184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:42.889892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:42.890493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:42.890995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:42.902199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:42.903876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:42.920295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:42.921466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:42.922531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:42.922531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:42.922531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:42.922531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 305/601] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:42.922531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:42.953578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af4eaf-4e59-447e-9d69-779837a70fef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.004072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.004655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.004935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.005211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/db771368757c449887be80383cceedbb/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.022776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=db771368757c449887be80383cceedbb, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.024306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.037768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:43.040492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 297/602] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:42 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/db771368757c449887be80383cceedbb/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:43.046867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:43.052091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d9a53-0cb0-4de4-b23b-7530e51c15a0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.124724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.125407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.125681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.125950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/2e5439bf8f1f480faef63316021d4b27/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.161607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=2e5439bf8f1f480faef63316021d4b27, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.163227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.192156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 306/603] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:42 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/2e5439bf8f1f480faef63316021d4b27/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:43.219529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:43.244446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dbc53-8715-4e95-a519-ab159bbbbb29 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.304244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.306440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.306920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.307401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.308246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:43.308720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.310305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076d84-9e46-44ee-bc93-c39a184529ff None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.326341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 298/604] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:43.332514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:43.355327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598271-9355-458a-bde7-952cccebd485 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.355952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598271-9355-458a-bde7-952cccebd485 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.356557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598271-9355-458a-bde7-952cccebd485 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.359171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 299/605] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:43.364574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.364894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.365121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.384918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:43.386177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:43.479237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208593fc-fd37-4c4f-b38d-a13baf11e31e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.505327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 300/606] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:43.520498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.521372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.521845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.522285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.529981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.531647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:43.542882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:43.543770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:43.544632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:43.546618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 307/607] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:43.555831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:43.640949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:43.676284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf7c1-e53c-4839-a714-95e440183951 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.710880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.711579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.711853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.712118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.712807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:43.713108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.714517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.726516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 301/608] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:43.731947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:43.745330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:43.825075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:43.874530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8f305-3eac-4293-98b5-3126898f5ebf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:43.911791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:43.912190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:43.912465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:43.912731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/8e742daf773049bf814cde4544b8a673/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:43.928796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=8e742daf773049bf814cde4544b8a673, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:43.931225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:43.963833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 308/609] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:43 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/8e742daf773049bf814cde4544b8a673/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:11:44.005171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83f759-4b97-4bc2-ba89-17ecae97278a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:44.056187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:44.063174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:44.064081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:44.064518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:44.064942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:44.085875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:44.091230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:44.092289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:44.093181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 302/610] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:44.098773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:44.187251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:44.233227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74de1b7-c9d3-45e4-8d30-e8eb36cb3098 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:44.273194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:44.274112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:44.274388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:44.274720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:44.275386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:44.275688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:44.277047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:44.286372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 309/611] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:44.293094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:44.423570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487d3ef-2557-44db-8471-8730365f9ed3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:44.473794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:44.474432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:44.474736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:44.475009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:44.486944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:44.488450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:44.510493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:44.511498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:44.512385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:44.512385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:44.512385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:44.512385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 303/612] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:44.517630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:44.669776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b3ad4-fc05-49aa-9ba1-c799100e0a90 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:44.701754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:44.717616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:44.718230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:44.718507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:44.718804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:44.719454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:44.719756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:44.721122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:44.737052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 310/613] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:44.743079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:44.818711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:44.895300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb390902-7ae9-4433-b9a1-d195758e195a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:44.933027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:44.933646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:44.933916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:44.934183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:44.941653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:44.943129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:44.954805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:44.956903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:44.959218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 304/614] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:44.964735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.121526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13739-e90e-4f5d-b8fd-a59b6b00d997 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:45.177106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:45.177731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.178005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.178274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.179086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:45.179391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:45.180893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a876df1-97b5-420d-93a2-62be41efae67 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:45.205845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 311/615] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:11:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:45.211449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.285316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4fe2-d10a-4d1f-9571-22961b30669d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.285713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4fe2-d10a-4d1f-9571-22961b30669d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.285986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4fe2-d10a-4d1f-9571-22961b30669d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.288399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 312/616] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:45.294822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.295096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.295362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.327582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:45.328859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:45.346518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9dc84-d0d7-4f5a-aec3-fe17f262bef9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:45.359129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:45.399817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:45.416172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:45.417149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.417637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.418121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.438177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 313/617] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:45.443519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:45.445176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:45.471938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:45.472872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:45.473758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:45.473758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:45.473758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:45.473758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 305/618] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:45.478754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.618567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a0505-e618-46e6-9092-a3c095ff821b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:45.668219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:45.668843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.669121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.669388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.670039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:45.670341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:45.671826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:45.684348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 314/619] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:45.689567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.744017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.784947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:45.859565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307be43-f040-4b3c-bd1c-5bf59b4b0a10 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:45.912144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b7126-37fd-4c51-ad14-57e3c75a01ff None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:45.916140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:45.917186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.917675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.918148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.927126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:45.929404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8706b71e-2d0a-4598-ab01-302a0ee48b97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:45.944618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:45.946094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:45.947446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:45.948969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 306/620] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:45.950151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3481cb-29f7-40dd-8edd-c9a6ecc13025 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.950422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3481cb-29f7-40dd-8edd-c9a6ecc13025 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.950714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3481cb-29f7-40dd-8edd-c9a6ecc13025 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.952045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 307/621] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:45.956100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:45.956253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3481cb-29f7-40dd-8edd-c9a6ecc13025 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:45.956344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.956618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.956888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.968571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ee7cfd0a-54e9-4ec8-be80-021be11b9db6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:45.974489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:45.975688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:45.976576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 315/622] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:45.977580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b73d0-d55e-4241-bdaa-e8b34b32c643 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.977580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b73d0-d55e-4241-bdaa-e8b34b32c643 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.977716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b73d0-d55e-4241-bdaa-e8b34b32c643 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.979022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 316/623] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:45.979988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060c86-a4cb-4a05-b350-07bf9ec0f4c7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:45.980326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060c86-a4cb-4a05-b350-07bf9ec0f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:45.980593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060c86-a4cb-4a05-b350-07bf9ec0f4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:45.981601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:45.999654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-16060c86-a4cb-4a05-b350-07bf9ec0f4c7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:46.000751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16060c86-a4cb-4a05-b350-07bf9ec0f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:46.071178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 317/624] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:46.072446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.072730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.072999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.079447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3481cb-29f7-40dd-8edd-c9a6ecc13025 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.094860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:46.096054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:46.111433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.118347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.119399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.119886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.120355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.121195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:46.121668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.124207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.146666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 308/625] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:46.148143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.178165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 318/626] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:46.178165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.290431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.330562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694669bd-8d8d-4805-a9f5-47999cc306f8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.350632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f16f-2843-458e-a2d7-be58f3d73da0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.388030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.388726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.389002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.389265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.389918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:46.390474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.392184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.394718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d479f-cb58-4458-a083-60722822228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.395481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d479f-cb58-4458-a083-60722822228c None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.395835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d479f-cb58-4458-a083-60722822228c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.396178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d479f-cb58-4458-a083-60722822228c None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/d763c798dc824e92b5e51deb0e059ea9/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.401519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 309/627] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:46.402117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.413432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d479f-cb58-4458-a083-60722822228c None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=d763c798dc824e92b5e51deb0e059ea9, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.416372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d479f-cb58-4458-a083-60722822228c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.430823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.436955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 319/628] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:11:46 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/d763c798dc824e92b5e51deb0e059ea9/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:11:46.438310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d479f-cb58-4458-a083-60722822228c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.561377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca6f46c-ea13-4fec-b295-ab98411a11dc None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.605751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.606541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.606940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.607302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.612795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d479f-cb58-4458-a083-60722822228c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.615896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.622220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:46.628063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:46.629289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 310/629] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:46.630375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.667242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.667884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.668165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.668441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.678666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.680234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:46.697623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:46.698645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:46.704508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:46.704508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:46.704508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:46.704508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 320/630] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:46.704508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.752169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.756258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f46084-0169-4fed-9e17-a2ea959c2fa5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.792834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.793461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.793742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.794016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.803357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38e464-468b-4384-ae04-1ba7ea7de5f5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.804483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b028f46a-5683-464d-8e52-110e31972108 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:46.810308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:46.811314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 311/631] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:46.817262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32062a-2679-4214-a5d1-2b71a50f8c60 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.818215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32062a-2679-4214-a5d1-2b71a50f8c60 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.818215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32062a-2679-4214-a5d1-2b71a50f8c60 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.818215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 312/632] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:46.818215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32062a-2679-4214-a5d1-2b71a50f8c60 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.844042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.844748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.845100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.845441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.846304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:46.846899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.848350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e026-8575-4b41-b0f4-b02af974725f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.862085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 321/633] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:46.862085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.862085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.862085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.876836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:46.878102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:46.884372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:46.933866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32062a-2679-4214-a5d1-2b71a50f8c60 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:46.946034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 322/634] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:46.947122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:46.970153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:46.970929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:46.971305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:46.971674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:46.972417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:46.972819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:46.974541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:46.992572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 313/635] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:46.992572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.009721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.072279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39bd36-805d-4530-a5f4-5f35bf55aa77 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.108612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.109575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.110016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.110426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.111251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:47.111680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.113275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.122445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 323/636] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:47.123508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.124233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9795538-0eb3-4084-8ce3-855210813d07 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.148204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.174346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.175400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.175782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.176171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.190741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.192949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:47.218416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:47.219573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:47.220679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:47.220679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:47.220679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:47.222609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 314/637] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:47.223497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.237517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ad4f3-6e04-430e-93cc-d7504eb9239d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.245134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.272733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.273405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.274174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.274440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.281398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.282840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:47.294933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:47.296094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:47.297364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:47.300549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 324/638] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:47.301283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.350209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefe57b-e75a-4d60-996f-31e1997dcdff None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.390846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.391839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.392444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.392963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.405238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.406781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.407733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1535e-04eb-4edd-b315-e3f246205f46 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:47.423372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:47.424463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:47.425473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:47.425473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:47.425473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:47.425473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 315/639] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:47.425704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.448514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.449124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.449641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.449641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.450793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:47.450793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.453012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.456934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.467666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 325/640] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:47.468514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.496296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.546066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849f1f1-23d2-48a4-b98c-475c951c5e21 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.570885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f19c-598b-4ed3-97ab-be8499a8c076 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.583895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.584985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.585270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.586358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.598955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:47.605348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:47.605817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:47.611887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:47.612941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:47.613696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.613696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.613696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.613696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:47.613991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 316/641] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:47.614078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.614159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.614248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.630653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 326/642] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:47.630653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.707461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.721688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.728829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013c6a-2270-404a-8220-fbb904d3d490 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.737893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320fc47-e577-483f-a500-9eb49203748d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.768167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.768983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.769407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.769835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.770468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:47.770811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.772189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.773793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.774432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.774735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.775005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.781893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.783356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.786097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 317/643] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:47.786757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:47.799073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:47.800046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 327/644] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:47.801139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.896426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75280a-54c8-44bf-ba90-5877b52ff685 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.908715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ea276-c29f-4008-84ae-cb97420aa038 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:47.934076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.934777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.935054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.935372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.936139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:47.936549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:47.937977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:47.941010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:47.941604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:47.941878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:47.942143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:47.951467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 318/645] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:47.952330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:47.955739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:47.957786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.013743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.015508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.056419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b244a8c-5487-4052-81a2-bbabab2cb021 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.090150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.090810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.091090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.091361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.099339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.101142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:48.113072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:48.114007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:48.114971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 319/646] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:48.115569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.124354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 328/647] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:48.124354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.174040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.182468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.223707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829e26c9-a7ce-4b8b-b654-5612759c198f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.231011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d97954-94bd-4dd3-8d7d-bf0c7a18d907 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.261819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.262430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.262725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.263002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.263710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:48.264012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.265686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.266513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.267228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.267453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.267720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.279317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.280749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.288739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 320/648] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:48.288739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:48.295998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:48.296937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:48.297872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:48.297872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:48.297872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:48.297872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 329/649] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:48.298513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.391863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.400414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1765e5-b5ef-4385-937e-c419f17f4f86 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.401343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1874a-605f-40c1-a8ec-4b2117283ad9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.426356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.436815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.437481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.437761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.438035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.441202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.441794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.442063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.442325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.443079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:48.443378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.444840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.454771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.456401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.465987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 321/650] 10.222.0.20 () {56 vars in 1137 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:48.469865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:48.471681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:48.472603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:48.473531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:48.473531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:48.473531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:48.477731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 330/651] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:48.478469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.580759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9c0d-f83b-4c74-9d74-394ea2a91a27 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.584894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb38ae4-6756-4209-bd53-951f2da01e67 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.616952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.617939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.618570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.619055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.619971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.620453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.620732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.621003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.621651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:48.621957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.623359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.635597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 331/652] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:48.636396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.647971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.670441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.690915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.692301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.733987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d6c82-7b6f-411a-9fe6-4b76c5a5c74d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.767451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.768051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.768331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.768603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.787658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 322/653] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:48.788482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.833526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.835018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e78c5-2971-4b9f-8704-0c502d35dabe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.844760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.868838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:48.888275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49164adf-a0d0-4e57-a2ca-494fff323725 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:48.921759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:48.922358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.922667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.922945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.923581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:48.923880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:48.925229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:48.931032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 332/654] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:48.932338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4ab9-ea57-416a-af88-b890f63343d0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:48.932624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4ab9-ea57-416a-af88-b890f63343d0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:48.932893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4ab9-ea57-416a-af88-b890f63343d0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:48.934735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 333/655] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:48.934879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4ab9-ea57-416a-af88-b890f63343d0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:48.937218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 323/656] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:48.937739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.034207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd345a0-9ffa-45dd-8f5f-68726f13e106 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.038027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4ab9-ea57-416a-af88-b890f63343d0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.047247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.067390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.068006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.068280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.068617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.071878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.072500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.072779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.073052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.077302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.079573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.080039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.081460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.081569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:49.094234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.095212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:49.096150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 334/657] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e030bfa1-694b-4a1f-8b41-f66034309bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:49.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.097572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:49.098441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:49.098441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:49.098441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:49.098629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.098728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 324/658] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.098728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.099683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.099683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.117509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:49.118737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:49.185702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 325/659] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:49.186523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.198244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceacd9-cf7e-4747-8bdc-651059d62dfd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.231906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.232534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.232809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.233076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.243501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.244972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:49.258631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.259583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:49.260501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:49.260501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:49.260501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:49.264471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 335/660] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.265559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.287285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.288308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b17fb-f83c-4f28-81ba-67bfa5d13b63 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.313231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.327036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.327783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.328161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.328527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.335536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.337624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:49.348721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.349785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:49.350880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:49.354960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 326/661] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.355771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.371235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eaeb4-9110-417a-b60f-176978185efb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.408694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.409299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.409577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.409843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.410633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:49.411000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.412418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.437342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 336/662] 10.222.0.20 () {56 vars in 1139 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:49.437943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.464562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78131063-c1a2-4472-b510-d9e1cd0d9cfd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.498367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.498988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.499258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.499523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.500155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:49.500452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.501796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.512730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 327/663] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:49.513710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.521000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.542473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f69e7-3dbf-4daf-97bb-ce283cec678d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.558117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.576147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.576754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.577035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.577314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.577951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:49.578254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.579652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.590462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 337/664] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:49.591281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.613917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78939278-8c12-42c5-8f8f-ae1bdf7cd920 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.647096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.647678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.647951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.648292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.649478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:49.649814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.651258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.662809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 328/665] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:49.663587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.692546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d4961-c679-4d3d-8123-b80d08a3a055 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.725422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.726022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.726332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.726737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.734970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.762253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e1e4d-e2d6-4d80-86a4-d231f885cc89 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.763786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.793397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.794837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.796531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.797143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.797421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.797688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.808084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:49.812817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 329/666] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.814963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.900282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 338/667] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:49.900282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.916438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fdd67-120f-4f17-9697-94d9fb520f52 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:49.934830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.950281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:49.950922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:49.951204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:49.951477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:49.958281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:49.959786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:49.970819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:49.971680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:49.972568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 330/668] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:49.973191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:49.996254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:49.998211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da19e-cbd7-4cc8-950f-7bca5e040a81 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.032829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.033462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.033742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.034013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.041030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.042476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:50.058557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:50.059603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 339/669] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:50.060899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.077509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346a876-63b8-48f3-90f1-9ed04dff4343 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.112874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.113464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.113737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.114001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.114744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.115047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.116505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.121932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 331/670] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:49 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.122737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.159860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.165209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a4a8d-567e-4073-a6d2-d612207f5733 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.204008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.204633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.204974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.205699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.205968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.206275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.207738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.221800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 340/671] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.221800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.224236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.242343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58c36a-0401-4620-83c5-6d381079c8d4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.283047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.283681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.283946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.284217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.295494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.296936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:50.312798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:50.313925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:50.314999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:50.314999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:50.314999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:50.314999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 332/672] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:50.314999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.335223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea28d-6a28-44c0-8409-7daafcb089fc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.376353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.377633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.377633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.377917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.378649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.380027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.380612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.399029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 341/673] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.399029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.421807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.432335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39b553-bbfd-49e4-a7e2-f98a941be528 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.471085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.471892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.472176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.472447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.473147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.473434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.474869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.482405 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.487861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 333/674] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.488706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.519533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96a7f-c384-4136-83f9-67ee1f780140 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.555230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.555944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.556216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.556482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.566863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.568363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:50.584524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:50.585636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:50.587148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:50.587148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:50.587148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:50.587148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 342/675] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:50.587549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.594488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a430d-927b-44f5-8d48-630fcc2e9af9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.632428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.633135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.633409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.633683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.634317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.634648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.636065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.652820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 334/676] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.652820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.663527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.694167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd51-184f-465f-b764-a6d131fdeefd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.718494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.731613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.732454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.732723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.732996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.733715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=db771368757c449887be80383cceedbb) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.734017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.735484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.741345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 343/677] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/role_assignments?user.id=db771368757c449887be80383cceedbb&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.742174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.784940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e95289-5b5b-47da-aa70-4c9d9c9f53d5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.829863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.830392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.831331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.831331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.831810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=2e5439bf8f1f480faef63316021d4b27) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.832119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.834052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.847033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 335/678] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/role_assignments?user.id=2e5439bf8f1f480faef63316021d4b27&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.848041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:50.862377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5695410-b981-471d-9496-027e89f88b40 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:50.922227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579744-5f34-449c-a247-b459ae100cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:50.922930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579744-5f34-449c-a247-b459ae100cce None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:50.923223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579744-5f34-449c-a247-b459ae100cce None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:50.923493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579744-5f34-449c-a247-b459ae100cce None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:50.924178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24579744-5f34-449c-a247-b459ae100cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:50.924479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24579744-5f34-449c-a247-b459ae100cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:50.925931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24579744-5f34-449c-a247-b459ae100cce None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:50.930979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:50.946557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 344/679] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:50.953542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579744-5f34-449c-a247-b459ae100cce None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.035530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be4b7f-c9f8-485d-a350-4be68c085a22 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.068179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:51.092725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.093284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.093560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.093815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.103235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.105413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.111436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579744-5f34-449c-a247-b459ae100cce None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:51.121247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:51.122143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:51.123070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:51.125922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 336/680] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:51.134785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.172062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.172975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.173396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.173808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.182261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.183966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:51.197711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:51.198954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:51.200109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:51.202358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 345/681] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:51.208985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.266088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90c1e7-7d02-4e00-8931-131c28c37a47 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.288802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:51.318118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.318750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.319017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.319287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.321607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:51.321607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.323193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.340092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 337/682] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:51.346255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.348949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99055b-b7dc-4b7c-bb56-b5d58e64096d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.395453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.396368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.397306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.397829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.399320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:51.400836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:51.401612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.403456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.431857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 346/683] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:11:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:51.437632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.509420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6310da-93c8-4ca3-8394-51628ae78c89 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.559210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.559928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.560201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.560468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.561264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=8e742daf773049bf814cde4544b8a673) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:51.561564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.563203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.572715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 338/684] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:51 2022] GET /identity/v3/role_assignments?user.id=8e742daf773049bf814cde4544b8a673&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:51.585389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b12183-5d9f-4893-8ce9-7872ed9e6c8a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.607234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:51.637199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:51.637829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:51.638104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:51.638368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:51.716824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:51.718321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:51.855720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 347/685] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 06:11:51.861080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:51.973814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b4c5b-dc5c-47f6-92a2-4baf08b875de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:51.982635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:52.017458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:52.018100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.018377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.018665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.036508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:52.037948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:52.056356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:52.057323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:52.058206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:52.058206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:52.058206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:52.062002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 339/686] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:52.063373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:52.102961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:52.236376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ebdbe-177a-40a9-bde8-fc06aa095707 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:52.281110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:52.282139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.282416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.282785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.283427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:52.283724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:52.285088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:52.297230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 348/687] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:52.302489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:52.442447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10a6ba-e6f0-45ed-af5a-8f073ef97d47 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:52.505705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:52.506249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.506524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.506902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.517302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:52.518752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2f1a7632-7a18-4434-bc56-2a7e61529c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:52.534417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:52.535327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:52.536199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 340/688] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:52.541870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:52.590482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3e23-37e1-407b-b4bb-4a73f18b1d01 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.590982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3e23-37e1-407b-b4bb-4a73f18b1d01 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.591258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f3e23-37e1-407b-b4bb-4a73f18b1d01 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.592995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 341/689] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:52.598156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a087d36-b650-41f8-8855-91673197a21b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.598430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a087d36-b650-41f8-8855-91673197a21b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.598873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a087d36-b650-41f8-8855-91673197a21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.620763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2a087d36-b650-41f8-8855-91673197a21b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:52.621956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2a087d36-b650-41f8-8855-91673197a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:52.639160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:52.700218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 342/690] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:52.711477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:52.757150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667d260-8b68-4e2f-98a1-49935e7183f9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:52.808970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:52.810021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:52.810465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:52.810953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:52.811938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:52.812241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:52.813677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:52.826364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 349/691] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:52.832157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a087d36-b650-41f8-8855-91673197a21b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:52.973707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a087d36-b650-41f8-8855-91673197a21b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:53.013624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:53.014552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.015322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.015782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.016714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=d763c798dc824e92b5e51deb0e059ea9) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:53.017211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:53.018842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:53.025634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 343/692] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:52 2022] GET /identity/v3/role_assignments?user.id=d763c798dc824e92b5e51deb0e059ea9&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:53.036203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:53.146751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5cc5cb-3f65-456d-bc10-e3499587a8c1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:53.184331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:53.184985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.185268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.185541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.187769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:53.198469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:53.204134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:53.205068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:53.205891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:53.208738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 350/693] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:53.215946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:53.243896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:53.358745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cd6cf-296f-451f-8724-d6213291e882 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:53.400297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:53.400966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.401254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.401581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.402265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:53.402567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:53.404022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219eb3-cf5c-4915-b75c-f643e9cdf6bf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:53.414311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 344/694] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:53.420071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:53.536019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd24735-a350-4a34-8e0a-b0f81eaa6395 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:53.571472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:53.572104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.572385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.572654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.583081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:53.584555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:53.593278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b61c-6928-4f3a-8fe8-8046ff7193e4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.593782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b61c-6928-4f3a-8fe8-8046ff7193e4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.594007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b61c-6928-4f3a-8fe8-8046ff7193e4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.595378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 345/695] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:53.600700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.600978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.601347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:53.604274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:53.605234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:53.606140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:53.606140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:53.606140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:53.606140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 351/696] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:53.611165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:53.626781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:53.627986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:53.670416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:53.718755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 346/697] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:53.734417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d886a35-c09d-4e8b-8664-3502b1b7592f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:53.769272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:53.769895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.770174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.770447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.771130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:53.771435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:53.772822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7ab00-bac5-412b-850c-9a1386728a6a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:53.785810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 352/698] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:11:53.791376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:53.795137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:53.800275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d2838-d7f5-4dff-9210-1fd20b8cfe2d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.800659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d2838-d7f5-4dff-9210-1fd20b8cfe2d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.800938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d2838-d7f5-4dff-9210-1fd20b8cfe2d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.802277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 353/699] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:53.807667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:53.807946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:53.808219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:53.827180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:53.828385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:53.897902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 354/700] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:53.973516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c98eb5-0319-46fb-b7af-b9534362914c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.020618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.021483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.021919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.022327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.032058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.033644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:54.050539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:54.051713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:11:54.052840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:54.054981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 347/701] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:11:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:54.060449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.071121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.128664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.138802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.181022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade4627-9317-4044-97bf-87d3b7d89c68 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.219429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.220285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.220664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.221046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.221792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.222461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1d823-124e-4e3d-b57b-9b00933c4101 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.223597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:54.236143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:54.237420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:11:54.238485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:54.245125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 348/702] 10.222.0.20 () {56 vars in 1129 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:54.245125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.278481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.279513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.280206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.281562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.282511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:54.283038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.284682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.308751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 355/703] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:54.310173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.353053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e758b-0a11-4f88-b5e9-13de31d1879a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.386407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.399438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.400768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.401261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.401757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.415229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.417300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.425560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.433296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cafee-4d8b-49d1-93a6-c1924c9b0f7d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:54.438291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:54.439263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:54.440132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:54.440132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:54.440132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:54.446874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 349/704] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:54.448430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.469341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.469959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.470237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.470554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.471248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:54.471931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.473070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.482631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 356/705] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:54.482631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.574822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f881ef6-310e-47c2-8465-2b9eed45f39c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.593320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf03974-f9a4-4f6f-8c6a-ed09fae32193 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.618005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.619330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.619647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.619888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.627645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.628253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.628526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.628792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.629423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:54.629718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.631102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.645488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 357/706] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:54.646762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.665142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.678480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.699984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.701368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398722-8fe2-496d-b533-e343f6ef65cd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.751161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b052c64e-31cb-479b-b5b7-4595286820eb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.787541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:54.788516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.789328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.789600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.790222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:54.790524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:54.792629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809cbb4-ac77-415f-b6c5-7194ab65e484 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 358/707] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcba79-7508-4cac-81c8-44417fc6cc82 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcba79-7508-4cac-81c8-44417fc6cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcba79-7508-4cac-81c8-44417fc6cc82 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 359/708] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:54.806690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcba79-7508-4cac-81c8-44417fc6cc82 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.815120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 350/709] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 18 06:11:54.816536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:54.816886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:54.817228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:54.838393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:54.839846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:54.902081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:54.932214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcba79-7508-4cac-81c8-44417fc6cc82 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:54.933463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 351/710] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:54.934688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:54.956658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:55.006033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.007307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.007625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.007903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.008564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.010012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.041849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 360/711] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:11:54 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:55.079940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28075f-0ad7-4736-82c5-beaacd988f6d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.119724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.120766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.121269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.121766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.134371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.136243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:55.153533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:55.154592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:55.155492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:55.155492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:55.155492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:55.157800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 352/712] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:55.163158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.256263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.308545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ac204-8b8b-4318-9884-cc692ee164d0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.331135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:55.348070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.349028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.349577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.350047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.351277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:55.351781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.353473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.362890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:55.371988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 361/713] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:55.382887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac00e3-e0a1-45e3-917c-03f6bee9ddc9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.384302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.421255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.422237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.423924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.424432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] PATH_INFO: `/v3/services/network` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.425983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.428888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:55.440092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:55.441200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:11:55.442027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:55.444787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 353/714] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:55.457601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.574203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0c0ea-22a2-440d-b508-447d17c307af None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.641614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f966e-0ff5-41a9-bb57-4f5a881eb701 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.646502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.647172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.647454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.647727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.661668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.663167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:55.683352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:55.684287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:55.685177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:55.697022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 362/715] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:55.697903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.702922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:55.721137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.721743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.722034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.722286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.722946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] RBAC: Adding query filter params (name=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:55.723244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.724612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.736638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 354/716] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:55.744402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.794035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:55.882190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51376608-3544-4a1e-aa78-825b3ae00e35 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.916194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4cf2c-b30c-49be-a254-59b466150332 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:55.944673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.945588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.946919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.947830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.949099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:55.949742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.952413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.970105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 363/717] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:55.976847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:55.977537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.977842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.978106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.978832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] RBAC: Adding query filter params (type=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:55.978964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:55.979134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:55.980671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c6a05-8e18-4e3c-bd55-b18a24713c72 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:55.988734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 355/718] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:55.990609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e67e-871b-485c-863e-9b7a43de09f9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:55.990907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e67e-871b-485c-863e-9b7a43de09f9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:55.991183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e67e-871b-485c-863e-9b7a43de09f9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:55.993452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 356/719] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:56.004117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:56.004440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:56.004675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:56.047160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:56.048387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:56.104422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:56.121478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f220df9-b56a-4974-aac6-e7868dd77b1f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:56.150773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 357/720] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:56.162979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:56.175025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:56.175813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:56.176047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:56.176390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:56.177229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:56.177534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:56.179089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:56.188767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 364/721] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:55 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:56.465668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:56.570752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08cb51a-6a13-4535-9d0e-05f99bb3f4d8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:56.615273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:56.615904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:56.616186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:56.616455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:56.632019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:56.637002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:56.637984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:56.638944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 358/722] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:56.644134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:56.807019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6dd34-53d9-4807-a47a-096ff2106fab None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:56.859841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:56.860460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:56.860737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:56.861004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:56.861674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:56.861973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:56.863370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:56.877319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 365/723] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:56.878641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:56.957096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:57.029862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719f4e0-5bd1-455d-8fbf-8acd1a237705 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:57.069471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:57.070073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.070348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.070637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.081372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:57.082847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:57.097939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:57.099048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:57.099928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:57.099928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:57.099928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:57.099928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 359/724] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:57.105423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:57.188483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:57.264644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90376964-c721-4774-8f7f-c006c81d7bfd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:57.312349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:57.312974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.313254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.313523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.314183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:57.314484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:57.316464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccc848-44d3-495b-a08b-277174c7ccaa None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:57.340686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 366/725] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:57.347785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:57.477908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a0bcf-dfd4-473a-8f3e-cc555cc62fef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:57.487045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:57.521617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:57.522243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.522563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.523706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.533484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:57.535054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:57.536903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390fd8-3f40-44f2-bba4-99065e74a5d9 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.537196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390fd8-3f40-44f2-bba4-99065e74a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.537470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390fd8-3f40-44f2-bba4-99065e74a5d9 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.541433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 367/726] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:57.545526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2317516-2caf-47db-881f-99fc5c16747b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.545810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2317516-2caf-47db-881f-99fc5c16747b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.546091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2317516-2caf-47db-881f-99fc5c16747b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:57.566894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:57.567886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:11:57.568850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:57.573293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 360/727] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:57.577189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:57.579306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f2317516-2caf-47db-881f-99fc5c16747b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:57.586854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2317516-2caf-47db-881f-99fc5c16747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:57.706223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 368/728] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:57.789232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1fba0-9b7e-4948-8da5-1ca04727dc78 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:57.844467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad3da8-1744-40e7-a268-10c4c8912dce None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.844907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad3da8-1744-40e7-a268-10c4c8912dce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.845319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad3da8-1744-40e7-a268-10c4c8912dce None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.846641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:57.846874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 369/729] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:11:57.847540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.847939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.848349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.849195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:57.849619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:57.851815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:57.852351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:57.852726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:57.853095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:57.871273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 361/730] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:11:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:57.879655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:57.881966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:57.888953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:57.906912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:57.947164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.017424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 370/731] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 18 06:11:58.019536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.037978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1e6ac-d464-40e6-a31f-36940d83fab1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.072195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.072789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.073061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.073325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.149135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.150701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.213571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1cc97-8564-4d15-9110-e8b3f7329d9f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.242973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.249782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 362/732] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 18 06:11:58.255178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.288619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.289146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.289427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.289702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.290359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.291763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.332488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 371/733] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:11:58 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:58.394499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.405714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e948d-643a-49a9-9207-a0082e0aafd6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.423035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.434174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.514310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1d3d-dfc1-4abf-8349-64af8a7a3856 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.543467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.544038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.544315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.544581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.551551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.552245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.552524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.552794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.563082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.564572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.581708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:58.588842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:58.589833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:11:58.590687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:58.591805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 372/734] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:11:58.595778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:58.598604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:58.599513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:58.600470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:58.600470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:58.600470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:58.602921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 363/735] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:58.606722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.730897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fc838-d418-4b0f-885d-24d541c5f7b3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.739671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e965fa-9761-4f7b-b745-e66572902787 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.740236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.778461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.779261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.779648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.780025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.780812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:58.781313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.783539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.784759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.785560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.785836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.786102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.787151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:58.787449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.789677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.796288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 373/736] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:58.807721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.810284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 364/737] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:58.818675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:58.867135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:58.946079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769db5f-5ae2-42f1-b073-c089237adc52 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.978849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96defe-c0af-4d67-bebb-56c35e547415 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:58.980173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:58.980803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:58.981106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:58.981391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:58.992863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:58.994315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:58.996175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-18205d99-7a2b-4fa0-a463-a8868574638b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:11:59.010852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:59.011816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:11:59.012707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:11:59.012707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:11:59.012707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:59.016803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 374/738] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:59.018043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d516b7-a342-4268-b56b-e4c0e85b0c46 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.018327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d516b7-a342-4268-b56b-e4c0e85b0c46 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.018614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d516b7-a342-4268-b56b-e4c0e85b0c46 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.020617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 375/739] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:59.022621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d516b7-a342-4268-b56b-e4c0e85b0c46 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.030672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.031305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.031580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.031846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.040249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.041756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-462645ec-950d-4a2d-9b70-03f6b058782f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:11:59.052984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:59.053829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 365/740] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:11:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9803-145b-4858-95e0-d9268cd92413 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.055517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9803-145b-4858-95e0-d9268cd92413 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.056132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9803-145b-4858-95e0-d9268cd92413 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.057052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 366/741] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:11:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:11:59.061830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.062108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.062376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.080954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:59.082163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:59.144580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d516b7-a342-4268-b56b-e4c0e85b0c46 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.166034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 367/742] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:59.166034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.185292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.186790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.187433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.187709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.188015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.188658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:59.188995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.190405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e309e-d1a5-4b8d-97a9-e99a601dd3c1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.208054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 376/743] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:59.208054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.208054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.208054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.224319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:11:59.225609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:11:59.289784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.292611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bef961-4f05-4f89-bf4d-42e1b3771f9c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.309507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 377/744] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:11:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 18 06:11:59.310447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.333693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.334626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.335152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.335653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.336841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:59.337791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.339817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.354055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 368/745] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:59.360769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.482520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541384-fad8-4b73-96af-e50d2666e93c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.509367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.541835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b4aea-ead4-405b-93bf-bf23ca965732 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.550107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.551808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.555163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.555770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.568243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.570849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.581074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.581681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.581958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.582229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.582994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=d763c798dc824e92b5e51deb0e059ea9) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:11:59.583303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.584740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.591522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 369/746] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/role_assignments?user.id=d763c798dc824e92b5e51deb0e059ea9&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:11:59.592371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:11:59.705932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:59.707043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:11:59.708088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:59.715934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 378/747] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:59.717214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.728690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72439b0-9318-44e8-8291-b0d5fab7448f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.771715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.778099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.779369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.779864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.780340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.781847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:11:59.783895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:11:59.799346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:11:59.814871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 370/748] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:11:59 2022] POST /identity/v3/services => generated 240 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:11:59.815780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:11:59.839472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b1c0e-eb29-45d5-8fa0-113ac44d8444 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:11:59.884559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:11:59.885261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:11:59.885543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:11:59.885865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:11:59.910143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:11:59.915138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:11:59.916129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:11:59.916946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:11:59.918222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 379/749] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:11:59.923976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.009923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe308b-f34a-4562-af4c-5f422235e6de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.057997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.061250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.061991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51a64a-8136-4513-a77f-6df12a319d28 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.062220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.062690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.063138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.064863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:00.066278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.067884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.091888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 371/750] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:00.098713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.106254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.106514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.106801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.107146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.107745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:00.108045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.109424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.118307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 380/751] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:00.123627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.178347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.246838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9dca-91f2-4ddd-8501-939096f9a671 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.273309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b975f9d-965e-44dc-9601-6a9f8482d182 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.284133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.284584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.284864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.285136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.354863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.356241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.375323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.375983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.376261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.376531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.403678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.405146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.417824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:00.444057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:00.445348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:00.446229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:00.446229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:00.446229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:00.446229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 381/752] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:00.451546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.481105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 372/753] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:00.486293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.500109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.589903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5c01d-da17-4d50-801f-c36a903c34f5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.624148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.624688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.624961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.625231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.642611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.642611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:00.659840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:00.660818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:00.661692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:00.661692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:00.661692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:00.667722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421736a5-2756-4664-b267-0dc64966a689 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.670320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 373/754] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:00.675851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.696263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.764208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.765080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.765366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.765632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.766275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:00.766570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.768025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.792759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f476a6-f0c1-460b-be65-9a375fdd739b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:00.797050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 382/755] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:00.802125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.836346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:00.836960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:00.837239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:00.837511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:00.838164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:00.838465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:00.839865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:00.867327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 374/756] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:00.875783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:00.956234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.980454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:00.998924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88513139-f75a-45f5-a143-2e86e526cc43 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.022566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f305562-4e53-4295-b16f-3fa6d833dfae None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.059945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.060698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.061059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.061331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.068590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.069286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.069462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.069739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.070070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.071025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.077988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.079503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-782ea601-d4c3-44da-a4d7-2f3e437d304f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:01.083513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:01.084570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:01.086277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:01.089912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 375/757] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:01.091333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c4b5-91e2-45f6-a978-7c6b3bf79347 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.091623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c4b5-91e2-45f6-a978-7c6b3bf79347 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.091899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625c4b5-91e2-45f6-a978-7c6b3bf79347 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.093151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 376/758] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:01.093834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c4b5-91e2-45f6-a978-7c6b3bf79347 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-82c5481f-3ab2-404e-acba-be42a4f1e7a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:01.095758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:01.097688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 383/759] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:12:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.099957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.101661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.121094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:01.122298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:01.202866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 384/760] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:01.204132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.204252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625c4b5-91e2-45f6-a978-7c6b3bf79347 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.233694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:01.247953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.248547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.249451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.249719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.250398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:01.250913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.253354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.280811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 377/761] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:01.286063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.316665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:01.386379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a79be-481c-470b-9369-b51c0470f4a9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.438994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b0e87-5673-4ab3-bf0f-b50cae3dea1a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.475470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.476445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.476918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.477377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.478301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=18bc18de709b424ca516b9082433f7ec) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:01.478858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.480661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.487006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 378/762] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/role_assignments?user.id=18bc18de709b424ca516b9082433f7ec&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:01.487310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.487934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.488213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.488483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.489229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:01.489533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.494673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.522093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 385/763] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:01.528479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.542645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.646055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:01.670719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba4878-4771-4805-8c0c-630e20e74736 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.707002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:01.723891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d5da1-0cdd-40ba-b942-18941f0b2d37 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.724319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.724968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.725246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.725518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.726141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.728782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:01.743706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:01.744732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:01.745753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:01.759184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 386/764] 10.222.0.20 () {56 vars in 1129 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:01.759184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.784702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:01.785393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:01.785670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:01.785939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:01.807272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:01.809479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:01.836626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:01.837902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:01.839025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:01.839025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:01.839025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:01.846747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 379/765] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:01.849562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:01.980252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48484761-3ec7-4232-b808-257fa4b2b471 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:01.988357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:01.988746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4a852-5f5c-4a08-ac69-bcd6b748a2d4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.030052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.030686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.030969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.031241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.031939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:02.032244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.033617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.064141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 380/766] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:02.071615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.072639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.073282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.073563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.073832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.074483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:02.074811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.076241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.090852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 387/767] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:12:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:02.098751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.196810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:02.229227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67484e5-0744-4426-860f-8b815a0ed032 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.263674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd2f66-6f4d-4e4c-a980-5428c6450173 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.274292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.274970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.275242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.275505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.285592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.287100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5d83bcb7-6d0e-45d0-addc-262f5d4ebe7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:02.302286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:02.303282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:02.304313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:02.308245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 381/768] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:02.309599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acf459-624c-4954-b019-8de1ddfccdd5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.309887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acf459-624c-4954-b019-8de1ddfccdd5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.310159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acf459-624c-4954-b019-8de1ddfccdd5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.312824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.313709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.314249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.314743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.315667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 382/769] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:02.315778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:02.316324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.316428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acf459-624c-4954-b019-8de1ddfccdd5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.318338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574242-c1af-4322-b2de-2fe2358d798c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.326785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 388/770] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:02.332174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.332295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.332863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.350314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:02.352067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:02.354204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:02.425207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 389/771] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:02.426488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.440216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acf459-624c-4954-b019-8de1ddfccdd5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.480840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:02.489827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.490530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.490904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.491247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.492659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:02.493031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.494801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d61ca5-c22f-47e3-b95f-77295f565118 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.507473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 383/772] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:02.509095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347ce6-82c5-4e05-8990-8543afce21d6 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.509444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347ce6-82c5-4e05-8990-8543afce21d6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.509783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04347ce6-82c5-4e05-8990-8543afce21d6 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.511105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 384/773] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:02.516559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04347ce6-82c5-4e05-8990-8543afce21d6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.541348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e87e0-da64-44b8-bbf0-ef2d9c8a0d91 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.582307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.583297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.583825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.584331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.585864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] RBAC: Adding query filter params (interface=public, service_id=2e9084f41de14d7d9764254452241f1e, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:02.586366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.588068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3724c-ea04-4200-9e03-8b806fdc4114 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.600514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 390/774] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/endpoints?service_id=2e9084f41de14d7d9764254452241f1e&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:02.602216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.602740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.603209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.622548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:02.624093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:02.632941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:02.655685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04347ce6-82c5-4e05-8990-8543afce21d6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.696424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 391/775] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:02.697558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.710898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.711821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.712354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.712873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.715955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=04ddfa59d1834ba3a7f1f27e963b090e, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.719599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.740549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 385/776] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:02 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:12:02.742332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:02.774253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:02.915744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efaaad-fbb7-47fd-98e6-5b755871c9e8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.945336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de7812-5981-4358-abc2-fc18fe262582 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:02.960426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.961319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.961596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.961863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:02.962465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:02.964163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:02.971220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 392/777] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/services => generated 772 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:02.990508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:02.991340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:02.991768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:02.992135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.004788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:03.011022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:03.012195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:03.013206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:03.018756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 386/778] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:03.021724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.028566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.038974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:03.199630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065ae2c-ab4e-4d84-ad32-80e34ca2c0d9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.221744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e093f-f5f6-473e-bab2-85c5b79506cd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.223359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:03.244672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.245649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.246090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.246518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.247321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.248936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:03.260504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:03.261563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:03.262389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 387/779] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:03.269793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.278119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.278819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.279044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.279315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.279990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:03.280289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.281818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.300073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 393/780] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:03.309984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.401060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:03.458904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2289aba-3765-4647-a335-e81a5db4cfe3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.471248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776994-32fc-4488-865b-c21289fdd552 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.514631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.514631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.514631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.514631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.515296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] RBAC: Adding query filter params (name=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:03.515296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.515296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.520717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 388/781] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:03.526250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.536742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.537485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.537857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.539346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.558865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.560496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:03.592372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:03.593363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:03.594247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:03.594247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:03.594247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:03.602489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 394/782] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:03.604005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.636617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:03.713943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c643b-5e68-4b36-a77c-a37de1db9e6f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.751698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.752320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.752811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.753080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.753710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] RBAC: Adding query filter params (type=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:03.754006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.755394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.756713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:03.763234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 389/783] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:03.768230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.789417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b979720-ea47-4a92-8734-8aabdb4cc053 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.863392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.864032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.864309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.864576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.865240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:03.865539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.867643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.895357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 395/784] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:03.903669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:03.904103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568749f9-0da1-40cb-8d78-a09ba6bcaf3d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:03.940544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:03.941160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:03.941438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:03.941709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:03.942408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] RBAC: Adding query filter params (interface=public, service_id=0cf54bb43a02405d8dfd68792bc42ca7, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:03.942754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:03.944121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:03.953954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 390/785] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/endpoints?service_id=0cf54bb43a02405d8dfd68792bc42ca7&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:03.959462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.026271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.055211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.064533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdbc09-d340-4e20-9aaa-444ecec279b7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.074179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f4cbf-ed8e-43d4-9b17-695f3948a009 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.098535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.099160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.099440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.099721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.100328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:04.101858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e1a19-0fe2-4026-9d10-682872af6590 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:04.108421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 391/786] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/services => generated 772 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:04.109530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028f60-1619-4ead-977c-612f1bf00bf1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.109858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028f60-1619-4ead-977c-612f1bf00bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.110102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef028f60-1619-4ead-977c-612f1bf00bf1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.117796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.118896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.119491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.119956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.120191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 392/787] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:04.125435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.125715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.125984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.130103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:04.132063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:04.147098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:04.148630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:12:04.149593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:04.157897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 396/788] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:04.159528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.175854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:04.177182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:04.281292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0be91-498c-48d0-8a67-f9db6999aa12 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.310718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 393/789] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:04.327880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.328525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.328806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.329079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.329839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:04.330150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:04.331666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:04.339122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.367166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 397/790] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:04.372915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.431975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.485887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f391fc1-a3b4-49c6-bd3f-bb4e7216c742 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.523495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.524131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.524409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.524737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.535187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:04.536689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:04.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:04.552896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:04.554550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:04.554550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:04.554550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:04.554550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 394/791] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:04.559703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.591735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.675467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e3b97-e308-4ce8-969e-26708b451a05 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.727209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952676-1222-43a9-8a81-da81880154bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.727818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952676-1222-43a9-8a81-da81880154bc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.728093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952676-1222-43a9-8a81-da81880154bc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.728360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af952676-1222-43a9-8a81-da81880154bc None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.728993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af952676-1222-43a9-8a81-da81880154bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:04.729288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af952676-1222-43a9-8a81-da81880154bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:04.730676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af952676-1222-43a9-8a81-da81880154bc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:04.752003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 398/792] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:04.759649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952676-1222-43a9-8a81-da81880154bc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.765384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1b6cb-d53a-4711-92bd-0cf81e8b12c5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.783136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.834233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.834918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.835736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.836007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:04.840525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:04.857221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:04.868879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:04.871066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:04.873035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 395/793] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:04.877575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:04.927611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af952676-1222-43a9-8a81-da81880154bc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:04.989244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:04.989884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:04.990157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:04.990421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.005750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.007279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:05.028690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:05.029643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:05.030530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 399/794] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:05.038297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b217bd-2afd-4112-ab6a-b6b525dce8e1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.040456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:05.112534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.113130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.113435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.113707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.114375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:05.114696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.116061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.132461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 396/795] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:05.140839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:05.204356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:05.228304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe75cd-c23c-463a-a0e8-9499a7462aad None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.255384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:05.299309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.299946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.300227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.300498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.301254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:05.301564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.303098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.324755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f73-3833-4bf6-8431-aaa6525020a7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.325565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 400/796] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:05.331862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:05.376920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.377480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.377815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.378088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.409560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.411217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:05.437694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:05.438719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:05.439637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:05.439637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:05.439637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:05.439637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 397/797] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:05.449045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:05.486411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5505f4e-c82b-448e-aea5-b6fe2dd62c90 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.520585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.521236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.521509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.521776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/d763c798dc824e92b5e51deb0e059ea9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.524206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=d763c798dc824e92b5e51deb0e059ea9, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.525639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc339a1a-a014-4950-acea-cb2bd397b538 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.540708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 401/798] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:05 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/d763c798dc824e92b5e51deb0e059ea9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:12:05.542093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3dac4-aff1-423d-9347-fe221ab981ad None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.542423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3dac4-aff1-423d-9347-fe221ab981ad None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.542725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3dac4-aff1-423d-9347-fe221ab981ad None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.545106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 402/799] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:05.546921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320df26-29d8-4f12-a9ea-20a2e29c2fbd None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.547198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320df26-29d8-4f12-a9ea-20a2e29c2fbd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.547513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320df26-29d8-4f12-a9ea-20a2e29c2fbd None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.553288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 403/800] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:05.553765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:05.555337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.555644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.555879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.581164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef20e19-8301-4b7f-8983-b4b8f78e3994 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.587683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:05.590450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:05.627804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.629333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.629855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.630343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.631519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:05.632529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:05.633866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.635828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5701ce-e892-4d28-9283-0d55e23f2703 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.660512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 398/801] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:05.660512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.660512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.660859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.681804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:05.683171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:05.701192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 404/802] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:05.702276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:05.785388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 399/803] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:05.822291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71799cd9-ee64-4883-835f-f7f405fef59d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:05.863081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:05.863715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:05.863986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:05.864251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:05.871532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:05.872972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:05.889294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:05.890484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:05.891452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 405/804] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:12:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:05.896380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.006101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.008125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.055274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.058321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58c5fc-bbf6-4388-9430-cc83445fd91d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.093564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.094161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.094437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.094754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.096329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:06.096647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.098068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.118425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 400/805] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:12:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:06.124072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.203617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da79830-2531-45d5-b297-f81b715b2ac7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.238898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f51b-5fee-4928-b1dc-cad73328ea2e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.240222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.240924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.241791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.242126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.242813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.244252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:06.253538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:06.254784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 18 06:12:06.255789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:06.261105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 406/806] 10.222.0.20 () {56 vars in 1129 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:06.261968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.272142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.282063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.283322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.283745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.284154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.298728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.301243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:06.320911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:06.322039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:06.322966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:06.322966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:06.322966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:06.326645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 401/807] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:06.327545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.369240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.387243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d8175-d6e1-4388-9365-f36057b3de24 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.433613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.434899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.435556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.435970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.437114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e72247-1ca0-4bcb-aeaf-a89efc98a33b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.457598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:06.468388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:06.469350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:06.470191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 407/808] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:06.470992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.487938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.488904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.489456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.490030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.490921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:06.491922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.493864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.504031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 402/809] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:06.505107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.558516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.602805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69e3b-3df8-47e6-9d23-c1d87902c23d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.637049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a6aeb-f58b-4b33-bf77-7e650f41e8aa None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.645083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.646059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.646515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.646998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.647947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:06.648439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.650038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.667640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 408/810] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:06.668847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.680159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.680759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.681032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.681298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.682231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.683060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:06.683269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.685488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.702047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 403/811] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:06.702047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.782705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb60b3-fae5-4278-8544-66eeba3943ed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.816195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04493e-4731-4247-b647-112e14209e5e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.817960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.818594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.818872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.819146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.820309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:06.820610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.821979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.826260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.827016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 409/812] 10.222.0.20 () {56 vars in 1124 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:06.827679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.858073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.858728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.859004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.859275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.865941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.867437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e1fb1a6a-4dae-4263-8d9a-d6ff71be391b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:06.879928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:06.880811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:06.881714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 404/813] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:06.882794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332b26-69a7-43f6-8b9d-f5f4722664f2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.883716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332b26-69a7-43f6-8b9d-f5f4722664f2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.883716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332b26-69a7-43f6-8b9d-f5f4722664f2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.884561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 405/814] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:06.885086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332b26-69a7-43f6-8b9d-f5f4722664f2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:06.908337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:06.940235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8185c-85fa-4aac-8ec7-74577a254dba None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:06.974919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:06.975595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:06.975858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:06.976155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:06.986924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:06.988374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.000182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332b26-69a7-43f6-8b9d-f5f4722664f2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:07.003014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:07.003974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:07.004878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:07.004878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:07.004878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:07.004878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 410/815] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:07.004878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.041011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.041638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.041917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.042188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.042879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.044248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa8cf6-095e-4ebb-8f90-0127c6d2dee9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.078434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 406/816] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:07.078434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.078434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.078434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.093474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:07.094790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:07.118825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8d35-8f1a-406f-a015-5da187ebe0d4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.122965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:07.152282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:07.178020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.179597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.180062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.180736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.182068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:07.182871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.184708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.186887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 407/817] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:07.188019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.205518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 411/818] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:07.207848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.344303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3c764-f596-474f-b43b-924ddfe572ea None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.380735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6f7f0-76b4-4873-86e4-568353e5e856 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.384701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.385799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.385799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.385977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/18bc18de709b424ca516b9082433f7ec/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.405880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=18bc18de709b424ca516b9082433f7ec, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.407389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.428199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 412/819] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:07 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/18bc18de709b424ca516b9082433f7ec/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:12:07.433164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.434455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.435037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.435536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.436416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:07.436927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.438684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.454832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 408/820] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:07.463717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.470720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.500825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:07.543741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:07.646604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100513bb-b6cf-4bf7-8263-dfa3f4926851 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.670993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea065c67-4d62-46d6-8e57-c49784f031a2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:07.696212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.696851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.697663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.697938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.708941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.711272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.712209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:07.712834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:07.713203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:07.713477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] PATH_INFO: `/v3/services/network` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:07.714148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:07.715529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:07.724525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:07.725472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:07.727119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:07.727652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:07.728564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 18 06:12:07.729388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:07.730136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 413/822] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:07.732638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 409/822] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/services/network => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:07.732815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.736018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:07.935541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:07.959104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41427e1-c4e0-4724-ba0b-0ca416773daf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.004418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.005035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.005313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.005582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.006217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] RBAC: Adding query filter params (name=network) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:08.006515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.007986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 414/823] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:08.025620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.056849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.089536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb128702-8e44-4c53-bdd9-657ff0c09a85 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.137558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.138279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.138551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.138845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.140074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:08.140374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.142592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00597e50-118c-4c40-80f3-cbc7500e1be3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.159792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41587982-eda1-4a63-939b-e6fcdb58517f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.168099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 410/824] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:12:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:08.173830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f094-bbff-45da-bb03-89f028d39ab2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.174179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f094-bbff-45da-bb03-89f028d39ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.174523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2f094-bbff-45da-bb03-89f028d39ab2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.176105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 411/825] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:08.176866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2f094-bbff-45da-bb03-89f028d39ab2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.207144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.207918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.208195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.208865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.209753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] RBAC: Adding query filter params (type=network) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:08.210115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.212165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c393824-997b-4820-a5f5-105f1b1b0148 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.223975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 415/826] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:08.226007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.227206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.227505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.253283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:08.254873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:08.273023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.298442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2f094-bbff-45da-bb03-89f028d39ab2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.338662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.339570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.339990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.340385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 416/827] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:08.340619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.341422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.345576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.456645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7144d-25c3-4f77-9950-eea9ff58a798 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.496678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.497970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.498246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.498514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.499491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.499651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.500870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.501309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.534826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 417/828] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:08.627537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 412/829] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:08.635780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.646041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.777537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.795254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf839e-9a6b-4cf3-9989-535d3346882a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.840523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.863910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.864546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.864822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.865087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.876139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:08.878469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:08.897865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:08.899763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:08.900800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:08.900800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:08.900800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:08.900800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 418/830] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:08.904878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:08.927159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d888e66-5314-4891-9e08-9eddcbcecdcb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:08.970214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:08.970963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:08.971242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:08.971508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:08.979187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:08.989353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:08.994932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:08.996110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:08.997137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:08.999434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 413/831] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:09.005381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.035282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edffbb-dc7d-45ea-9d71-577a8927fbed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.074071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.075223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.075974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.076435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.077287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.077765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.080077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.098656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 419/832] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:09.105345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.135059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.144503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac00b9-02aa-4802-9ff0-b54e77f566f3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.190248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.190881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.191155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.191419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.192239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.192633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.194495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.204810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 414/833] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:09.212184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.233360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e57ea-1e43-49e5-8cbb-80267428a02c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.248319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.272848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.273293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.273564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.273828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.280999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.282445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:09.295659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:09.296553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:09.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:09.301437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 420/834] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:09.302907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.327825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341536d1-b642-41a5-8163-1fc653332e22 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.365201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.366752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.367347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.367814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.379547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.381430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:09.397941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:09.398948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:09.399831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:09.399831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:09.399831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:09.406343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 415/835] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:09.407522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.412637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d4e5a-505e-4449-9c01-677b7f7843f2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.421159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.451207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.452193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.452612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.453033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.453865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.454401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.456258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.475143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 421/836] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:09.476902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.511037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.519924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2692cb3-c78a-47ad-8239-c1d69cd420e0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.561716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.562403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.564596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.564943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.565666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.566038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.567614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.586341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 416/837] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:09.587378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.591963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932422ba-5f1e-4594-96de-63fbf320ae5a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.629044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.629953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.630425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.630893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.632433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.632928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.634650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60111245-76a3-45fb-afed-36652db48b9b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.646723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 422/838] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:09.657717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81adc6-cdcf-42a2-afc8-e454dba837aa None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.658053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81adc6-cdcf-42a2-afc8-e454dba837aa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.658263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81adc6-cdcf-42a2-afc8-e454dba837aa None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.659945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 423/839] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:09.665119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.665415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978535b-73fb-4a7e-9464-72bbf917e704 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.665696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978535b-73fb-4a7e-9464-72bbf917e704 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.665966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978535b-73fb-4a7e-9464-72bbf917e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.688102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7978535b-73fb-4a7e-9464-72bbf917e704 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:09.689337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7978535b-73fb-4a7e-9464-72bbf917e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:09.700574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040654ae-7bfb-4408-b97d-58927a5e0089 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.741275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.741959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.742235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.742524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.751573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.753105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:09.768132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:09.769203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 18 06:12:09.770274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:09.771420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 417/840] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:09.775669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.777688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 424/841] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:09.873421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e0df5-78f9-40d0-bf2b-dd79b804b8ce None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.874018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e0df5-78f9-40d0-bf2b-dd79b804b8ce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.874018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e0df5-78f9-40d0-bf2b-dd79b804b8ce None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.875277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 425/842] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:09.880256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.880534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.880795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.888728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23ecef-67fe-4a11-af63-889bfff6c102 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:09.900861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:09.902343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:09.919987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.926430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:09.928364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:09.929015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:09.929523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:09.929712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:09.931492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:09.931492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:09.933803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:09.953813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 418/843] 10.222.0.20 () {56 vars in 1135 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:09.960946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:09.985757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 426/844] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:10.062952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.067900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db18eb-4e04-4c7e-b218-58bfb05c57ce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.103193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.103801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.104074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.104345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.164652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.182359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.184069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.195188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b749c-966d-485d-b4ea-130b15960b0c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.233142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.234315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.234843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.235202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.235998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.237566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:10.248034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:10.249035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:10.249859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:10.254275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 427/845] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:10.255206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.279788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.295088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 419/846] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:12:10.295927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.369163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df62d2-622f-4516-809f-d74054652d05 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.388466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.403093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7da719-b4a5-4925-93c3-a15a3d415b6c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.404188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.404795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.405074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.405345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.405966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:10.406268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.407700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.412200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 428/847] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:10.413496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.443587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.444748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.445068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.445342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.458674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:10.464209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:10.465140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:10.465977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 420/848] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:10.466706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.507959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.526540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da59a0b-2d95-4bc2-9255-ce47ee3c70de None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.559653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.560596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.560872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.561162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.571801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.573247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.579831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d1b1b-7a2b-44ed-8563-6c5d382df754 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:10.587133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:10.588260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:10.589197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:10.589197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:10.589197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:10.589197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 429/849] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:10.589421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.622056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.623006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.623435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.623851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.624715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:10.625223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.626585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.631321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 421/850] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:10.643811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.662810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.719148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5aad8c-bfd1-43bf-8e78-80c92e1645da None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.768445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.769508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.769848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.770122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.771191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:10.771491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.774271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.777237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb8de3-1c36-47a0-8d41-6cb530d57445 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.777374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.791016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 430/851] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:10.794452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.830061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.830952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.831385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.831653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.833160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:10.833160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.833918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.847006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 422/852] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:10.852496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:10.913889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c61b8-ecef-452c-aff6-e680f692bccb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.951196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:10.952374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:10.953214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:10.953696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:10.954108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:10.967608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:10.969499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:10.986168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:10.987397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:10.988465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:10.988465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:10.988465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:10.988683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522015e-0495-4881-a648-854bc75418e6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:10.995036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 431/853] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:10.996063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.037696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.038924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.040208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.040634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.049452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.051717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.055296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:11.067707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:11.068817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:11.070060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:11.077257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 423/854] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:11.079329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.123247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f58ee-d69c-43eb-a524-f0010ad58971 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.158354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.159069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.159436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.159891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.160536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:11.160971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.162378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.179909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 432/855] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:11.180996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.210051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d722db-fe63-4baa-8d3d-4d979293d782 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.246128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.254756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.255861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.257201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.258093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.259404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:11.260232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.262517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.281526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 424/856] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:11.282647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.310312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdf19c-1dd8-48b3-ae37-47887679c916 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.323429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.350002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.351879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.352362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.353391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.363521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.366045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:11.382833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:11.383965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:11.385081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:11.392649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 433/857] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:11.393977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.398276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c879f2-99aa-4d40-a8d9-bdf7bc919e6a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.443266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.443911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.444240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.444511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.445240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=d763c798dc824e92b5e51deb0e059ea9) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:11.445538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.447047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.456470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 425/858] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/role_assignments?user.id=d763c798dc824e92b5e51deb0e059ea9&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:11.528737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499dada-1104-4527-83ac-c486602cf881 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.565724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.566341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.566712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.566984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.567718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:11.568019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.569498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.590728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 434/859] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:11.603735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.668981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.714131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64408963-f0b1-42d2-b61e-885ca7d35b8e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:11.748300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:11.748894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.749164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.749429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.818364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:11.819779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94094a7-fe2e-4895-9d9b-016386f5a181 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:11.878901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.930326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 426/860] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:11.936361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:11.966878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:11.983837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df49c4-98b1-4ac4-a3d2-ef2ffdf801ca None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.984346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df49c4-98b1-4ac4-a3d2-ef2ffdf801ca None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.984818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df49c4-98b1-4ac4-a3d2-ef2ffdf801ca None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:11.986315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 427/861] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:11.991431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:11.992020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:11.992729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.013126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:12.014324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:12.052485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83df673-506c-477a-a091-82ec0f7cdf4c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.091523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 428/862] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:12.091714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.092147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.092520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.092869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.108911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.110765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:12.126737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:12.127718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:12.128632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:12.128632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:12.128632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:12.130664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 435/863] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:12.138594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.224649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:12.241539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833634c7-28d3-4efd-a750-f9cd2a953337 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.277966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.278836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.279282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.279699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.280488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:12.280995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.282493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.295143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 429/864] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:12.300728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.356969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.385168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:12.405286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d0133-ed3e-4d2d-a22a-297e5cbe3c4c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.440027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.440790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.441116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.441453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.450196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.451727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:12.470909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:12.471862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:12.472907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:12.474994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 436/865] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:12.480518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.492882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a621e-bcd8-4dc7-81b7-421a1077b813 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.529175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.529799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.530072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.530339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.530998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.532598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:12.541993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:12.543136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:12.544258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:12.550720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 430/866] 10.222.0.20 () {56 vars in 1132 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:12.551984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.564936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:12.626518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d857f-2f27-4ed0-9613-a89d47d2ae36 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.663291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57e754-a5e4-4f85-aaf4-6e17f4118c2a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.676458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.677857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.678285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.679245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.680331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:12.680785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.682810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.696865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:12.700066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.700804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.701231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.706247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.706247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:12.706247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.706247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.707216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 437/867] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:12.708612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.715449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 431/868] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:12.716646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:12.818747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:12.823401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa97e6-e25e-4b59-9b9d-f236fd5dddbf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.856105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787f85c-4793-42c8-8cfb-dd6e51d84710 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:12.867479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.868245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.868685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.869054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.870249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=18bc18de709b424ca516b9082433f7ec) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:12.870706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.872863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58af9-7f46-476b-9f9a-af89aba31982 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.886704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 438/869] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/role_assignments?user.id=18bc18de709b424ca516b9082433f7ec&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:12.902212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:12.903076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:12.903455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:12.903837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:12.904615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:12.905026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:12.906646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef95e75-06dc-4768-80a4-95d838743c35 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:12.918789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 432/870] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:13.018375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7e4a9-a60a-49d9-b2e5-5a3041e10f55 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:13.018963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7e4a9-a60a-49d9-b2e5-5a3041e10f55 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:13.019153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7e4a9-a60a-49d9-b2e5-5a3041e10f55 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:13.021119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 439/871] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:13.027913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d29d8-a2a9-425b-8dd9-31b032b7403d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:13.028357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d29d8-a2a9-425b-8dd9-31b032b7403d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:13.028741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d29d8-a2a9-425b-8dd9-31b032b7403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:13.049825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b34d29d8-a2a9-425b-8dd9-31b032b7403d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:13.051367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b34d29d8-a2a9-425b-8dd9-31b032b7403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:13.135074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 433/872] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:13.406132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7e4a9-a60a-49d9-b2e5-5a3041e10f55 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:13.485185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:13.527824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7e4a9-a60a-49d9-b2e5-5a3041e10f55 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:13.564798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:13.565407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:13.565685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:13.565952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:13.566690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:13.568160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7f92b-2bf2-464c-a777-b47199d5d31f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:13.587294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 440/873] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:13 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:13.799784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af975-1e16-420c-8e39-4c0935c48ca1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:13.800277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af975-1e16-420c-8e39-4c0935c48ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:13.800277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af975-1e16-420c-8e39-4c0935c48ca1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:13.801836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 434/874] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:13.807745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:13.808277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:13.808754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:13.828703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:13.830332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:13.919902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 441/875] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:13 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:14.201441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af975-1e16-420c-8e39-4c0935c48ca1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:14.271823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:14.312569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af975-1e16-420c-8e39-4c0935c48ca1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:14.346930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:14.347550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:14.347824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:14.348085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:14.348664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:14.350018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:14.359429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:14.360411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:14.361244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:14.362014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 435/876] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:14.373361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:14.425917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:14.481247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5195a-f1b3-406a-afc0-36e08c4d3d66 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:14.519774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:14.520985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:14.520985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:14.520985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:14.521542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:14.521845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:14.523400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96619974-36b1-4538-927e-a185da1fc5e8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:14.528423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 442/877] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:14.534118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:14.641660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c7c61-4bc0-42b4-815b-55dfee1d2f74 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:14.670826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:14.676046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:14.676649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:14.677337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:14.677964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:14.683317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:14.683788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:14.685284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d7282-607a-4b48-ace6-6c685458c461 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:14.690702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 436/878] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:15.215331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ec555-1742-4d8d-95f1-c061cb3d3e9b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.215819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ec555-1742-4d8d-95f1-c061cb3d3e9b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.216202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ec555-1742-4d8d-95f1-c061cb3d3e9b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.217915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 443/879] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:15.223334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.223808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.224077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.243042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:15.244222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:15.248334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13d8b-6ff1-408c-b83a-58268fbb0aac None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.248869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13d8b-6ff1-408c-b83a-58268fbb0aac None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.249348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13d8b-6ff1-408c-b83a-58268fbb0aac None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.251152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 444/880] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:15.256372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.257514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.258062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.277149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:15.278342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:15.322690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 437/881] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:15.354906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 445/882] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:15 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:15.592998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:15.622553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:15.674941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:15.697594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea1c9-81d7-40f7-8480-29f97fc2b39e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:15.730733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:15.737867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c183a-84d5-4fd3-9812-3f5ca143e801 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:15.740714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:15.741983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.742324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.742702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.743420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:15.744851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:15.768038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 438/883] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:15 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:15.799563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:15.800216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.800540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.800811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.813151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d82ee473-d3c2-4e17-8215-d1c8a3b314d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:15.818546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:15.820193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:15.821643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 446/884] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:15.826520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:15.836238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e6dc-b372-42fc-9eed-988545617f52 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.836542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e6dc-b372-42fc-9eed-988545617f52 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.837396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e6dc-b372-42fc-9eed-988545617f52 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.839142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 447/885] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:15.844880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.845123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.845422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.865018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:15.866414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:15.952241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 448/886] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:15.962765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b714d-e3cb-422b-bb22-49131e62d323 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:15.997233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:15.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:15.998101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:15.998367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:15.999056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:15.999353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.000700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.009243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 439/887] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:16.015157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.051402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.123778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65ad8-113e-4890-8115-9b6d195eb0bc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.158628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.159258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.159534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.159804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.169972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.171454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:16.186541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:16.187512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:16.188433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:16.188433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:16.188433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:16.188433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 449/888] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:16.193583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.222981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.307290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.310454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.321313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54cb25-3b59-4aa5-b7c3-af2f03f04eb6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.330489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287b6ed-10f8-4573-a773-d21be00f3ddf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.367566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.368701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.369126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.369713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.370835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:16.371570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.373225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.376623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.377336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.377698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.378071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.378785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.380304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:16.393174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:16.394453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:16.395357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:16.396538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 440/890] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:16.397705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.399329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 450/890] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:16.400327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.533828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22a8ab-dfd1-4c00-864c-12ebf8284a7c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.538423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f9cf-a5f4-4f49-9494-062d342d09b8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.545721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.577924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.578545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.578849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.579125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.585832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.587044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.587418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.587772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.588240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.588856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.589376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:16.590024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.592679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.598060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 451/891] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:16.602559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:16.603533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:16.604481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:16.607551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.608068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 441/892] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:16.608876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.623459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.726041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94df9f-50b7-4fa0-a281-ffcd2085118c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.733598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ace561-22d1-4852-96ed-2af31ea28537 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.763034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.763654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.763962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.764180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.765070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:16.765377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.766802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.776950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 452/893] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:16.778658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.789324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.790255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.791207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.791700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.793342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:16.793917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.795901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.821665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 442/894] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:16.827533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.862423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.865547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:16.891960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc557c-fc4f-4e95-8407-f7275ce0dbdc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.933007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.933618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.933894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.934163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.934899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] RBAC: Adding query filter params (interface=public, service_id=04fc3e95065c48feaf7991db9df1db7a, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:16.935209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:16.936580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c3663-6447-4599-bf2e-b3b768909f27 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:16.941474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 453/895] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/endpoints?service_id=04fc3e95065c48feaf7991db9df1db7a&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:16.942823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d9e1-fc9b-402f-be28-b7589df1ec6b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.942953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d9e1-fc9b-402f-be28-b7589df1ec6b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.943187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260d9e1-fc9b-402f-be28-b7589df1ec6b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:16.945232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 454/896] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:16.949415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260d9e1-fc9b-402f-be28-b7589df1ec6b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:16.956270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b568b-bc53-4f32-b843-188273e96dc2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:16.995425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:16.996484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:16.997387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:16.998079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.051384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260d9e1-fc9b-402f-be28-b7589df1ec6b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.073769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.075248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.090765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.091377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.091886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.091886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.092489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.093841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c7674-bfa7-4df8-b166-16f994808c76 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.099677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 455/897] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:12:17.100732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913083b-075e-4972-82b0-13fc131be892 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.101010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913083b-075e-4972-82b0-13fc131be892 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.101273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913083b-075e-4972-82b0-13fc131be892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.108186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:17.119442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:17.120886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8913083b-075e-4972-82b0-13fc131be892 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:17.122134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8913083b-075e-4972-82b0-13fc131be892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:17.217533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 456/898] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:17.226616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 443/899] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:16 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:12:17.229390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913083b-075e-4972-82b0-13fc131be892 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:17.337148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913083b-075e-4972-82b0-13fc131be892 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.369310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:17.375105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.376086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.376560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.377030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.388987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.391170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:17.406118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:17.407087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:17.407962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:17.407962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:17.407962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:17.410003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 457/900] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:17.416284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:17.496510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:17.527055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc70c0-9b8e-4f88-b6c7-24427012fcfa None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.555862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:17.565596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.566540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.567139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.567608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.568614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:17.569551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.571278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.588905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 444/901] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:17.599759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:17.619530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d380f6-c6de-48e9-b414-2606e5d3a418 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.663514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.664960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.665451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.665909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.666902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.668615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.687060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 458/902] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:17 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:12:17.721188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81208e5-297a-4902-870c-69afcb4c4466 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.775163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.775954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.776242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.776553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.785360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.786971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:17.800460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:17.801429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:17.802297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 445/903] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:17.808423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:17.861717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:17.922382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf728f-34c8-4023-bdb1-45399ac39a77 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:17.961541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:17.962218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:17.962499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:17.962789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:17.963494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:17.963796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:17.965250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f26f2-3d02-471e-9bcd-cbe9ab05a5b0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:17.978973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 459/904] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:17.984734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:18.030901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:18.055279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714fa4-f547-4915-9e81-71a064251714 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.055580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714fa4-f547-4915-9e81-71a064251714 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.055839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a714fa4-f547-4915-9e81-71a064251714 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.057435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 460/905] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:18.062437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.062758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.063032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.082268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:18.083545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:18.104634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c132-ba00-4a6a-acbc-d02d60602982 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:18.151733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:18.152803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.153224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.154032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.155249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] RBAC: Adding query filter params (role.id=fed9876de0fd463fa75d25a2e6ce7fe7, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=18bc18de709b424ca516b9082433f7ec) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:18.155560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:18.157048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:18.163822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 461/906] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:18.164485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 446/907] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:17 2022] GET /identity/v3/role_assignments?user.id=18bc18de709b424ca516b9082433f7ec&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=fed9876de0fd463fa75d25a2e6ce7fe7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:18.445057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:18.491580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:18.564374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f44cc0-2fe6-4afa-adf0-2141950dcb52 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:18.599083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:18.600580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.601083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.601547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.602327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:18.603921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:18.612924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:18.613853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:18.614755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:18.617463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 462/908] 10.222.0.20 () {56 vars in 1132 bytes} [Sun Sep 18 06:12:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:18.623755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:18.733077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a4ce7-ad87-4c2e-b456-8e78aba22784 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:18.740402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:18.769124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:18.770067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.770538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.771025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.771884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:18.772406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:18.773943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:18.779336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 447/909] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:18.786617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:18.885470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:18.890403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93340-0857-4a68-9704-babebff9fa49 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:18.928976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:18.929830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:18.930271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:18.930719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:18.931504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:18.931945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:18.933459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:18.939234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 463/910] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:18.947713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:19.048467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e303bd-623a-4520-9bda-fa0c00e1b81e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:19.086233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:19.087208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.087683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.088161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.089155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] RBAC: Adding query filter params (interface=public, service_id=28af075cc087498f8dc1b929bf5de34d, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:19.089653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:19.091224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d8191d-ab3b-4ae0-b2b5-11c9b8f958f5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:19.097076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 448/911] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:18 2022] GET /identity/v3/endpoints?service_id=28af075cc087498f8dc1b929bf5de34d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:19.103897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:19.213048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e87859-3a6b-4dd3-8268-974aa63376a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:19.248101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:19.249160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.249657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.250144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.250992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:19.252601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b84772-50d9-4209-90ef-330843e727e0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:19.260596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 464/912] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:19 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:12:19.379206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db788c-270e-48fb-a0e4-15e8b570db07 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.379822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db788c-270e-48fb-a0e4-15e8b570db07 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.380303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db788c-270e-48fb-a0e4-15e8b570db07 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.382796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 449/913] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:19.388420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8315-019f-4f42-b27a-eb814f2389f5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.388769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8315-019f-4f42-b27a-eb814f2389f5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.388975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8315-019f-4f42-b27a-eb814f2389f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.409508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-04aa8315-019f-4f42-b27a-eb814f2389f5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:19.410779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-04aa8315-019f-4f42-b27a-eb814f2389f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:19.433984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:19.490833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 465/914] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:19.759803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db788c-270e-48fb-a0e4-15e8b570db07 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:19.797661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:19.870017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db788c-270e-48fb-a0e4-15e8b570db07 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:19.904479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:19.905102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.905667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.905935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.906512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:19.907884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:19.917056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:19.918037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:19.918909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 450/915] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:19.919707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901e6-d1dd-4544-b021-b03a8f14678b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.919707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901e6-d1dd-4544-b021-b03a8f14678b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.920176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43901e6-d1dd-4544-b021-b03a8f14678b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.922571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 466/916] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:19.926252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:19.926712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:19.926996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:19.927271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:19.946780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:19.947972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:20.022815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 467/917] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:20.035269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:20.039365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f0c08-cafb-4078-ac48-f0d25b9da01e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.074305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.074927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.075199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.075462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.076201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:20.076499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.077855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.086893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 451/918] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:20.088990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.183566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:20.201719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3c151-8489-4e43-a47c-3bd3cf438821 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.237247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.237853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.238131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.238403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.239060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:20.239364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.240788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.245947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 468/919] 10.222.0.20 () {56 vars in 1114 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:20.251238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.324538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.358588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2442a4-4726-4a51-9cc7-422d285c2c78 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.396159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.396803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.397121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.397391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.398086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.399640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ebc99-2639-43c9-97eb-71cdd988dcbe None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.424580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 452/920] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:20.472763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236a43-c4bf-4f04-a7c2-3661e4df1947 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.516916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bf2b2-5152-4a63-95ce-5f353f8999b1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.517549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bf2b2-5152-4a63-95ce-5f353f8999b1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.518016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bf2b2-5152-4a63-95ce-5f353f8999b1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.524125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.524125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.524125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.524125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.524558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.524558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.524959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 453/921] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:20.526941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.527475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.527803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:20.534546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:20.535659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:20.536527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:20.539369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 469/922] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:20.551352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.553131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:20.554454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:20.568907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:20.571581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:20.649204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 454/923] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:20.692280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f76cf-2a26-4df4-ba68-a7168f041e90 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.727490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.728102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.728382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.728649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.729276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:20.729588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.731009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.736238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 470/924] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:20.743859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.847839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755e265-1b62-41b8-be88-6d8dec1f6b82 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:20.884374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:20.885433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:20.885920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:20.886384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:20.887250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:20.887791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:20.889415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:20.895475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 455/925] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:20.901602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.925725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:20.944221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:20.996250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.034316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752456fd-9b33-47ad-85d4-ba4a36c70991 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.043302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c337a-3415-45da-9e92-5f3d80ca728c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.079807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.080589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.080794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.081083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.081456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.081881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.082477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.082979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.084121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] RBAC: Adding query filter params (interface=public, service_id=1a42d870b6fd4f1dbb04af1c35d3f385, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:21.085034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.086685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.097853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:21.098036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 471/926] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/endpoints?service_id=1a42d870b6fd4f1dbb04af1c35d3f385&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:21.099116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:21.101396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:21.102311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:21.103174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:21.104474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 456/927] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:21.110379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.185988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.225828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f15d02-f0ba-41cb-900c-2f05cf085b0f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.244493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517e34d-92e9-40f9-8c6f-09c9993743cb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.277112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.278327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.278799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.279234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.280054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:21.280594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.282127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.282390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.283003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.283379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.283761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.284503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.285988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70944f2d-1859-4fe5-884c-0e0615cb4632 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.293351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 457/928] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:21.298622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 472/929] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:12:21.307240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.432632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8c3c-306b-4b03-a29d-4ffdef2bf1a4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.478953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b008-1721-4056-923c-8872f20f9668 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.479264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b008-1721-4056-923c-8872f20f9668 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.479497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494b008-1721-4056-923c-8872f20f9668 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.483121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.483749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.484024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.484295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.484690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 473/930] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:21.486411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.487256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.487691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.487978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.496927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.497710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.499383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.510561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:21.512496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:21.515065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:21.515949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:21.516834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:21.516834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:21.516834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:21.517174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 458/931] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:21.526308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.608343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 474/932] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:21.646031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f048c-aa62-48d5-8fcd-a4d9ce0af98b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.680779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.681433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.681714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.681985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.682656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:21.682955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.684324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.697691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 459/933] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:21.703486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.808992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4297fab-84e7-4956-ab9a-042249ae3503 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:21.839846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.846562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:21.847313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:21.847591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:21.847864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:21.855404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:21.856854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:21.868666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:21.869582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:21.870942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 475/934] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:21.875854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.914474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:21.926970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:21.983517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfe460-9edc-4f6b-bb99-d393651265af None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.016161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49524b01-7347-48a2-81da-350cf92582b8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.021493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.022709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.022988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.023256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.023984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:22.024284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.025708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.049276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 460/935] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:22.050190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.055114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.055990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.056470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.056949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.057733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.059364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:22.068625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:22.069630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 18 06:12:22.070514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:22.077991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 476/936] 10.222.0.20 () {56 vars in 1132 bytes} [Sun Sep 18 06:12:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:22.079496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.139480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.160942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3e963-1912-4dd4-b775-5fbac53528f0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.163799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.188684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa59e11-3563-47a6-978e-54ec7e2392c6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.200318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.201340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.201587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.201960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.213673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.215435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.227376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.228228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.228656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.229063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.229886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:22.230337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.231889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:22.232148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:22.233248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:22.234310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:22.234310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:22.234310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:22.241185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 461/937] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:22.246361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.246681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 477/938] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:22.248245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.354958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52dec98-7aa7-4da0-a06a-ca26fe39443f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.355253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31f1f8-1c50-4533-b977-ea2e4e149aa4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.375680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.391741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.392637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.393105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.393573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.394598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:22.395142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.399072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.401799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.402995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.403496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.403956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.404791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:22.405368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.407013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.408847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 478/939] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:22.409865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.423639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 462/940] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:22.430121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.454047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.547982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758073b-c348-40eb-91f8-e420fa5db074 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.560517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e02fe-d43d-43a3-b0b4-2ae2986f19d9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.594709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.595352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.595640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.595913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.596984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.598449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.619926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.620909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.621375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.622177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.630694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.632420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.634738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 479/941] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-cd308a6d-1ba4-4c4a-ab12-3e7ecee08358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:22.650551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:22.651550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:22.652437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:22.654760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 463/942] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:22.660737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.705732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418d645-3dd1-434c-a873-2824509a2505 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.706090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418d645-3dd1-434c-a873-2824509a2505 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.706271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418d645-3dd1-434c-a873-2824509a2505 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.708012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 464/943] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:22.713396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.714220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.714669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.714794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.742498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:22.744476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:22.791260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557a7d-058e-44c6-bd1c-239b584f6792 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.828674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.829653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.830159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.830646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:22.831560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:22.832057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:22.833687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:22.835080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 465/944] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:22.847831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 480/945] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:22.854072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:22.866292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:22.957951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6721109-cb90-4e6a-9cbb-a063edaf7024 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:22.995836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:22.996882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:22.997354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:22.997820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/18bc18de709b424ca516b9082433f7ec/roles/fed9876de0fd463fa75d25a2e6ce7fe7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.000289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=18bc18de709b424ca516b9082433f7ec, role_id=fed9876de0fd463fa75d25a2e6ce7fe7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:23.001923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:23.019277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 466/946] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:22 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/18bc18de709b424ca516b9082433f7ec/roles/fed9876de0fd463fa75d25a2e6ce7fe7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:12:23.095988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:23.205386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:23.211987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e561f3-68f3-4c7f-b074-6e1eb16b6589 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:23.249342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:23.249949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:23.250221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:23.250489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.251183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:23.252613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:23.255207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING keystone.server.flask.common [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 18 06:12:23.275494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.sql.core [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 06:12:23.275494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 18 06:12:23.275494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [parameters: {'id': 'cc06f0db90ef469ea2d8bb9433d5ccb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 18 06:12:23.275494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81397) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc06f0db90ef469ea2d8bb9433d5ccb2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc06f0db90ef469ea2d8bb9433d5ccb2. Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:12:23.283076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 18 06:12:23.283884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application yield resource Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise self.value Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application session.commit() Sep 18 06:12:23.284990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.flush() Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application compat.raise_( Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 06:12:23.286069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 06:12:23.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:12:23.288383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [parameters: {'id': 'cc06f0db90ef469ea2d8bb9433d5ccb2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:23.289243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:23.292379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc06f0db90ef469ea2d8bb9433d5ccb2. Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:23.293314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 481/947] 10.222.0.20 () {60 vars in 1140 bytes} [Sun Sep 18 06:12:23 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:23.293851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:23.362697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:23.399420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3daa76-2602-485f-b2aa-277daa527934 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:23.435571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:23.436220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:23.436494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:23.436760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.445271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:23.446773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c1332ec7-e5a1-4ee6-9b10-9055d54156dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:23.459777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:23.460730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:23.461617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 467/948] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:23.469178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:23.558988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae919-7674-4972-81a8-bda2e9abd2e0 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:23.559328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae919-7674-4972-81a8-bda2e9abd2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:23.559547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae919-7674-4972-81a8-bda2e9abd2e0 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.561152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 468/949] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:23.566147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:23.566424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:23.566732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.580878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c198c-1253-4547-b93f-93d1d2fee3a6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:23.590063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:23.591488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:23.623333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:23.623960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:23.624297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:23.624571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:23.625801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:23.626105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:23.628226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:23.654074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 482/950] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:23.679373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 469/951] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:23.949122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:23.966682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:24.067716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6be38-16e2-4f44-b450-58e397ee8cae None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:24.103940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:24.105638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:24.105638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:24.106170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:24.106977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:24.108534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:24.118038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:24.119029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:24.120014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:24.122832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 483/952] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:24.130617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:24.186799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:24.230548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b51f91-7d34-46a3-9b90-d77ff4a75c6c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:24.266089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:24.266719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:24.266991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:24.267256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:24.267873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:24.268167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:24.269521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:24.278708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 470/953] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:24.280232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:24.360609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:24.385432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2bcd-21ec-43ad-bf8b-ea18c92872c0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:24.421478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:24.422092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:24.422364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:24.422657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:24.423292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:24.423591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:24.424969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143cd503-0d16-4075-a201-cba7e688bca9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:24.434697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 484/954] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:24.436371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:24.543728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ae2d9-5289-4f8a-a400-2154d8c43d0f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:24.564799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:24.580125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:24.581043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:24.581550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:24.582097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:24.583007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:24.585288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d72f1d3-c4c1-46c6-953c-b642831924be None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:24.609346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 471/955] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:25.069182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2112d5-8933-4ee7-b8f1-32de5aa3bf40 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.069555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2112d5-8933-4ee7-b8f1-32de5aa3bf40 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.069733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2112d5-8933-4ee7-b8f1-32de5aa3bf40 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.071379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 485/956] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:25.076656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f2c1-1cdf-4e8b-82f8-a6e2fca61839 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.077210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f2c1-1cdf-4e8b-82f8-a6e2fca61839 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.077679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f2c1-1cdf-4e8b-82f8-a6e2fca61839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.097064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0060f2c1-1cdf-4e8b-82f8-a6e2fca61839 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:25.098565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0060f2c1-1cdf-4e8b-82f8-a6e2fca61839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:25.174298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 472/957] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:25.334802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f1c1-86c2-4002-b066-435ccd6f222f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.335294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f1c1-86c2-4002-b066-435ccd6f222f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.335294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319f1c1-86c2-4002-b066-435ccd6f222f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.336870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 486/958] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:25.342090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.342373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.342664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.361434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:25.362629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:25.432470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 473/959] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:25.456027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f1c1-86c2-4002-b066-435ccd6f222f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:25.570305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319f1c1-86c2-4002-b066-435ccd6f222f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:25.604973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:25.605532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.605804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.606067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.606667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:25.608019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:25.616931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:25.618495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:25.620835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 487/960] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:25.624122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:25.656830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:25.719144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:25.724798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:25.735980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1abdf0-1c0e-4aaf-84f1-1a7b15075187 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:25.773527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:25.774162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.774440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.774744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.775400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:25.775695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:25.777141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:25.783458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 474/961] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:25.788690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:25.849864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6789e7c-4fc0-4a1b-8e02-cf612ad5c0a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:25.890135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:25.890711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.890983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.891247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.902713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:25.907111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7db954fc-b987-4624-8db2-71bc6039acd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:25.907965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:25.908893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:25.909775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 488/962] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:25.910489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367648ad-6712-4384-8733-c30e761504cb None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.911082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367648ad-6712-4384-8733-c30e761504cb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.911082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367648ad-6712-4384-8733-c30e761504cb None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.912341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 489/963] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:25.916898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367648ad-6712-4384-8733-c30e761504cb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:25.920588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93abc-a62e-43c1-8247-a8294bcd0b24 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:25.956327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:25.956924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.957198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.957464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.958083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:25.958431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:25.959909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb5220-36a8-48cc-9b44-1f53caf3ffe0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:25.974646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 475/964] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:25.974646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:25.974646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:25.974646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:25.992209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:25.993420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:25.997084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.039750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367648ad-6712-4384-8733-c30e761504cb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.086641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 476/965] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:26.086845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.086845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.086845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.086845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.087170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:26.087558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.088906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.098448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 490/966] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:26.106600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.204977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.214535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d80a3-bcce-4d3b-aa36-e4833dace48b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.248812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.249438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.249716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.249986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.260589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.262022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:26.276886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:26.277851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:26.279282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:26.279282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:26.279282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.279282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 477/967] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:26.283808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.394437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca48-e122-4910-a9f2-52430dfbe33a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.402395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.423600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.432930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.433530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.433808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.434076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.434804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:26.435205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.436678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.450122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 491/968] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:26.456451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.516852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4badc-b7a0-4d98-833c-22288bc0b562 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.556028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.556773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.557049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.557315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.558071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.559520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.562160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING keystone.server.flask.common [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 18 06:12:26.573377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25f1bb-34c8-423d-88ea-cbc844f704e9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.577130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.sql.core [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 06:12:26.577130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 18 06:12:26.577130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [parameters: {'user_id': '184c40d4af3b4ed884b61b02a6e38a90', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 18 06:12:26.577130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result.read() Sep 18 06:12:26.598192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:12:26.599285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application yield resource Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application next(self.gen) Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.session.rollback() Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.force_reraise() Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise self.value Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application session.commit() Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 18 06:12:26.600361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._prepare_impl() Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.session.flush() Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._flush(objects) Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application compat.raise_( Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application flush_context.execute() Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rec.execute(self) Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 18 06:12:26.601336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 18 06:12:26.602203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result = self._query(query) Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application conn.query(q) Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result.read() Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 18 06:12:26.603175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [parameters: {'user_id': '184c40d4af3b4ed884b61b02a6e38a90', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:26.604857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 18 06:12:26.606315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 478/969] 10.222.0.20 () {60 vars in 1131 bytes} [Sun Sep 18 06:12:26 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.616134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.617170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.617170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.617170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.617170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.625542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.627199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:26.641038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:26.642956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:26.644400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 492/970] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:26.650459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.712371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.726406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a8eda-2a45-4228-814b-a307ee02d1e9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.772117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.773024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.773402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.773788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.780209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579ae1b-2608-4d25-8de2-19cbb5ea3799 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.784389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.786150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:26.800063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:26.801937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:26.803941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 479/971] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:26.808600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.822339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.823000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.823276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.823548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.824282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:26.824587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.826019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.845502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 493/972] 10.222.0.20 () {56 vars in 1149 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:26.850945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:26.883003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.926180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2a392-f5e9-40b5-9cc2-4e413c891e45 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.953767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bad4ec-83a1-4544-b989-3a6ff1a8e493 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:26.957437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:26.963762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.964432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.964610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.964854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:26.965958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:26.966263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:26.971606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9437722-c2bd-4526-908d-ba1f07d53204 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:26.994330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:26.994973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:26.995248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:26.995512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.012990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 480/973] 10.222.0.20 () {56 vars in 1101 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:27.014335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259a71-dbcf-4c02-af61-96e15136db93 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.014912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259a71-dbcf-4c02-af61-96e15136db93 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.015228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05259a71-dbcf-4c02-af61-96e15136db93 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.016624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 481/974] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:27.026625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.026625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.026625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.043741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:27.045404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:27.082282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.084775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.130755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 482/975] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:27.198512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 494/976] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:26 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:27.203961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.266476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.316977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18080514-87b9-4d40-bb39-2fbcaee63faa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.359059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.359692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.359968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.360240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.371488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.372932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:27.388562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:27.389756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:27.390720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:27.390720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:27.390720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:27.390720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 483/977] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:27.396219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.399599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.421493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.505509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425a3d2-1b74-4d0a-bc45-0a725afba580 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.530332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dd2a0-7088-4285-9b10-739f52f86fd5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.533443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.542132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.542773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.543054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.543312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.543952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:27.544249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.545607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.561962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 495/978] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:27.562842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.568168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.568980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.569285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.569587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.570168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.571796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:27.580733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:27.581680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:27.582513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 484/979] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:27.588255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.629237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.672277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71f46-24e7-415c-a161-647be537a03e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.689531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18353a4-61d4-4f5e-965c-f91413394086 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.707934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.708593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.708887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.709183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.716216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.717756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.724390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.725554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.725830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.726095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.726740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:27.727039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.728391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:27.729839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:27.730727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:27.731642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 496/980] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:27.736107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 485/981] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:27.736905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.743108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.775263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.836622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a8079-a44c-414a-b3d1-674cb7fcbbdb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.850628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068fc00-0c5e-4949-97a5-3ae5a44c6775 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:27.859931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:27.871174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.871789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.872066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.872343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.873027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:27.873335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.874793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.887203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 497/982] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:27.891167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:27.891764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:27.892037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:27.892382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:27.892926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:27.893224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:27.893643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:27.894598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:27.900241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 486/983] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:27.906292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:28.000899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f87fbe-4595-461d-9387-2f754cb6fb35 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:28.002810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:28.009296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279894bb-1452-4eb7-b751-e651902a6f00 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:28.043197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:28.043800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.044130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.044341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.045052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] RBAC: Adding query filter params (role.id=fed9876de0fd463fa75d25a2e6ce7fe7, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=18bc18de709b424ca516b9082433f7ec) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:28.045353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:28.046166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:28.046795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.046895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:28.047071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.047336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.048034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] RBAC: Adding query filter params (interface=admin, service_id=1a42d870b6fd4f1dbb04af1c35d3f385, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:28.048335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:28.049664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525a04-a4a9-4398-8283-380b4ca5a97d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:28.052785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 498/984] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/role_assignments?user.id=18bc18de709b424ca516b9082433f7ec&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=fed9876de0fd463fa75d25a2e6ce7fe7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:28.054916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 487/985] 10.222.0.20 () {56 vars in 1249 bytes} [Sun Sep 18 06:12:27 2022] GET /identity/v3/endpoints?service_id=1a42d870b6fd4f1dbb04af1c35d3f385&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:28.064524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:28.120263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:28.180690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d0cf9-b7f0-4676-804a-f87679628126 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:28.217311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:28.217936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.218213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.218480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.219122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:28.220512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253eab5-46f3-4e7c-ae82-04c521734aab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:28.227292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 499/986] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:28 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:12:28.265400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa3b-1df0-4496-ae14-2a172354517b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.265670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa3b-1df0-4496-ae14-2a172354517b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.266117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725fa3b-1df0-4496-ae14-2a172354517b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.267533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 488/987] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:28.274231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff2cb-1f94-4ced-aeca-c40805f2a703 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.274531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff2cb-1f94-4ced-aeca-c40805f2a703 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.274826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff2cb-1f94-4ced-aeca-c40805f2a703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.295905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-896ff2cb-1f94-4ced-aeca-c40805f2a703 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:28.297483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-896ff2cb-1f94-4ced-aeca-c40805f2a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:28.376139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 500/988] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:28.643848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725fa3b-1df0-4496-ae14-2a172354517b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:28.748103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725fa3b-1df0-4496-ae14-2a172354517b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:28.783207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:28.783800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:28.784076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:28.784345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:28.785008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:28.786362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6a12a-b5e1-4795-846b-11649b5002a1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:28.801367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 489/989] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:28 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:30.257720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddddc5-48dc-4da9-bea8-4e61af771c5d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.258250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddddc5-48dc-4da9-bea8-4e61af771c5d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.258250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddddc5-48dc-4da9-bea8-4e61af771c5d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.259804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 501/990] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:30.265861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab539a-ea9f-4cb0-8cdb-2e25d36ad6b6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.266338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab539a-ea9f-4cb0-8cdb-2e25d36ad6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.266520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab539a-ea9f-4cb0-8cdb-2e25d36ad6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.286348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-eaab539a-ea9f-4cb0-8cdb-2e25d36ad6b6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:30.287646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eaab539a-ea9f-4cb0-8cdb-2e25d36ad6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:30.306160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:30.377924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 490/991] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:30.438950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188af43-5d61-4890-a37a-8f5b2ab10219 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.439262 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188af43-5d61-4890-a37a-8f5b2ab10219 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.439497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188af43-5d61-4890-a37a-8f5b2ab10219 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.440982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 502/992] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:30.446077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.446371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.446673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.468283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:30.469477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:30.538822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 491/993] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:30.643860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188af43-5d61-4890-a37a-8f5b2ab10219 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:30.688094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:30.745557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188af43-5d61-4890-a37a-8f5b2ab10219 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:30.779399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:30.780024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.780297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.780562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] PATH_INFO: `/v3/services/image` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.781140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:30.782492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:30.791258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:30.792326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:30.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.794970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 503/994] 10.222.0.20 () {56 vars in 1105 bytes} [Sun Sep 18 06:12:30 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:30.798217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:30.809212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:30.883273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:30.909461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fe687-2bc5-494f-ba00-5af64b20d138 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:30.922543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32057b6c-daee-4ed3-9745-9516560b72ba None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:30.930806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:30.947065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:30.947669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.947997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.948259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.948887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:30.949191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:30.950562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e67419-d28d-42ec-acbf-e840df3405c2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:30.958625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 492/995] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:30.960079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f8af3-aaa2-4957-9ae7-3076fdac78d4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.960556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f8af3-aaa2-4957-9ae7-3076fdac78d4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.960900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f8af3-aaa2-4957-9ae7-3076fdac78d4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.962155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 493/996] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:30.962875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f8af3-aaa2-4957-9ae7-3076fdac78d4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:30.966294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:30.967139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.967422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.967693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:30.968274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:30.969638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-0427dfba-922f-47fa-af5c-5927d8017643 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:30.978907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:30.979967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 18 06:12:30.980796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:30.984928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 504/997] 10.222.0.20 () {56 vars in 1126 bytes} [Sun Sep 18 06:12:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:30.985983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:30.986269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:30.986542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.003805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:31.005011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:31.062752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f8af3-aaa2-4957-9ae7-3076fdac78d4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.069537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 505/998] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:31.070416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.098964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.099631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.099923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.100170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.100799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.101102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.102460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.107062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 494/999] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:31.186109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.190197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1dc9db-a018-4a75-a65e-d9afa2d17831 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.228992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.229616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.229894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.230163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.230833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.231135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.232536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.243408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 506/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:31.245068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.249193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.338347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.355811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90836e94-4f43-42a6-a483-8ed493ec12c6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.392277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.393381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.393656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.393920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.394540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.394865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.397232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.402759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 495/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:31.408008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.450027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304f2a-b3c4-4390-910d-158f53492836 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.483015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.486383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.487054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.487329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.487594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] PATH_INFO: `/v3/services/compute` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.488167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.489645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:31.498700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:31.499768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:31.500672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 507/1002] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:31.507128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.523800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e353419-8f09-4e92-87af-04afe996a495 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.538045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.559319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.559919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.560192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.560518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.561114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.562455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.582292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 496/1003] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:12:31.623721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf50c293-95e2-4011-9ba0-264f7002c5fe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.669548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.670174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.670449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.670745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.671384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.671687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.673073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.684005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 508/1004] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:31.685150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.798184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb2dc1-5307-42aa-9442-cc51f1f19b1f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.824765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.834992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.835583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.835857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.836121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.836746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.837042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.838390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.848236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 497/1005] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:31.850211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:31.939046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:31.952967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5a15-63af-467e-8456-1bfb1d77e0e7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:31.985808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:31.986430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:31.986740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:31.987009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:31.987722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] RBAC: Adding query filter params (interface=public, service_id=286231c0fa5842658d69d8dc52d54437, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:31.988031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:31.989446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6c6b4-ef85-4f50-9349-2e8ddc6878a5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:31.996056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 509/1006] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:31 2022] GET /identity/v3/endpoints?service_id=286231c0fa5842658d69d8dc52d54437&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:32.001863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:32.104802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d3734-16c2-4e74-b733-755bac00c568 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:32.138055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:32.138676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:32.138953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:32.139223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:32.140074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:32.141426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736dbefd-d09b-43ad-8533-026652aa01fa None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:32.154849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 498/1007] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:32 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:12:33.267435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf882-6b47-47ac-a377-ab6cee20689e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:33.268009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf882-6b47-47ac-a377-ab6cee20689e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:33.268009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43cf882-6b47-47ac-a377-ab6cee20689e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:33.269358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 510/1008] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:33.275179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd3f63-4601-4eef-9c44-dfd580093b80 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:33.275588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd3f63-4601-4eef-9c44-dfd580093b80 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:33.275780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd3f63-4601-4eef-9c44-dfd580093b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:33.294234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5efd3f63-4601-4eef-9c44-dfd580093b80 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:33.295443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5efd3f63-4601-4eef-9c44-dfd580093b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:33.345725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:33.364406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 499/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:33.631472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cf882-6b47-47ac-a377-ab6cee20689e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:33.736637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43cf882-6b47-47ac-a377-ab6cee20689e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:33.745322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02144-331f-4b6e-b727-d07b3da63f44 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:33.745700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02144-331f-4b6e-b727-d07b3da63f44 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:33.746061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e02144-331f-4b6e-b727-d07b3da63f44 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:33.747536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 500/1010] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:33.752425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1be1e-857e-4a7a-b92f-8a0a2a081956 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:33.752704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1be1e-857e-4a7a-b92f-8a0a2a081956 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:33.752975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1be1e-857e-4a7a-b92f-8a0a2a081956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:33.774438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-91e1be1e-857e-4a7a-b92f-8a0a2a081956 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:33.775689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-91e1be1e-857e-4a7a-b92f-8a0a2a081956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:33.780780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:33.781391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:33.781667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:33.781936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:33.782616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:33.783993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:33.800350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 511/1011] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:33 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:33.856896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 501/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:33 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:34.130718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:34.147676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:34.249574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ca734-fc5e-4517-b47b-f2fee8a8d353 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:34.267924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673da325-62e7-4ae7-a0ce-a8260503667e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.268248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673da325-62e7-4ae7-a0ce-a8260503667e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.268472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673da325-62e7-4ae7-a0ce-a8260503667e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.270127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 502/1013] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:34.275561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.276139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.276139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.285928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:34.286569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.287429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.287692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.288844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:34.290299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:34.293573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 512/1014] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:34.294453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:34.295733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:34.297565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:34.363966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 503/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:34 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:34.379031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:34.400107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d779e6-b898-485b-8e3f-2ebe7226bc58 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:34.434147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:34.434777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.435054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.435320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.435955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:34.437441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:34.477868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 513/1016] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:12:34 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:12:34.627793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:34.674218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:34.736290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415a6be-bea9-4da0-a003-7438f78388c7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:34.770203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:34.770860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.771181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.771454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.772038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:34.773401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:34.782025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:34.783159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:34.784003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 504/1017] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:34.788962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:34.895706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70617191-0d06-4016-8a40-e38c54015d68 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:34.924895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:34.932044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:34.932738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:34.933389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:34.934036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:34.934928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:34.935557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:34.936937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:34.946843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 514/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:34.947975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:35.049271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233e459-2d5f-406b-836a-d74a757dba59 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:35.056460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:35.084910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:35.085511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:35.085790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:35.086062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:35.086712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:35.087017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:35.088375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1252b729-6bde-404b-a1c9-3a65f3dbe356 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:35.097831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 505/1019] 10.222.0.20 () {56 vars in 1112 bytes} [Sun Sep 18 06:12:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:35.098978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:35.198524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697b848-7390-49f8-aa98-cbc9e0644374 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:35.231878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:35.232493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:35.232773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:35.233045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:35.233705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:35.235191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e6072-d2ef-44c5-8351-8b1bdffc78ef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:35.262614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 515/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:35.967308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db16ef-f569-468c-b845-c55e6f805162 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:35.967859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db16ef-f569-468c-b845-c55e6f805162 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:35.967859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db16ef-f569-468c-b845-c55e6f805162 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:35.969311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 506/1021] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:35.974423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:35.974863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:35.975044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:35.992790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:35.993976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:36.042411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:36.070248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 516/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:36.341684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db16ef-f569-468c-b845-c55e6f805162 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:36.442428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db16ef-f569-468c-b845-c55e6f805162 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:36.482461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:36.483107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.483381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.483647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.484215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:36.485559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:36.494394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:36.495495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:36.496346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:36.498900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 507/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Sun Sep 18 06:12:36 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:36.501407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:36.607947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1a7a-6a9b-4c2e-8522-16d8f696cfc1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:36.642455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:36.643091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.643532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.643808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.644438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] RBAC: Adding query filter params (name=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:36.644741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:36.646172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe42797-024e-44c1-8f7a-0423bb1a6541 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:36.655432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 517/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:36.656953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:36.692934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:36.693572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac714-6c46-4141-845e-a0e740b80b29 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.694701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac714-6c46-4141-845e-a0e740b80b29 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.695051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cac714-6c46-4141-845e-a0e740b80b29 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.698619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 518/1025] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:36.702655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.703128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.703985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.725517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:36.726775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:36.766361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30325-9464-41fb-bebd-233dd0c4e4a3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:36.766594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:36.799826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 519/1026] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:36.805064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:36.805665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.805940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.806207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.806864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] RBAC: Adding query filter params (type=image) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:36.807168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:36.808520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:36.818948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 508/1027] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:36.819774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:36.920279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbe699-7ab7-4f77-800f-fa99f8e28bd3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:36.963228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:36.963855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:36.964136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:36.964410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:36.965121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] RBAC: Adding query filter params (interface=public, service_id=881df01f4d7e46e5ad86ca47daf2f31e, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:36.965430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:36.966844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:36.977576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 520/1028] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:12:36 2022] GET /identity/v3/endpoints?service_id=881df01f4d7e46e5ad86ca47daf2f31e&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:36.978785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:37.063909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:37.098562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:37.103702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb77e6-bc28-4357-a931-00a07b4f2442 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:37.109530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:37.146673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:37.147287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:37.147569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:37.147843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:37.148449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:37.150342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:37.158212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 509/1029] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:12:36 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:12:37.200215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cf897-35b1-4a84-b8fe-726e5aaa8b33 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:37.238539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:37.239219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:37.239493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:37.240290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:37.241429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:37.242905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535c103-e64f-41f2-a957-f4713e50f5b4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:37.244691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 521/1030] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:37.250508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:37.367405 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ab550-b155-4435-88a9-7271a918fb5c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:37.419598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:37.420209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:37.420487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:37.420754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:37.421422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:37.422872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef13d7-55e3-4e84-a6ed-2e9cca01c97d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:37.450155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 510/1031] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:12:37 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:37.507085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957344d-9d46-4c64-bd12-6e2ba1c48a19 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:37.507441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957344d-9d46-4c64-bd12-6e2ba1c48a19 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:37.507627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957344d-9d46-4c64-bd12-6e2ba1c48a19 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:37.509283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 522/1032] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:37.515085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:37.515377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:37.515634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:37.535466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:37.537238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:37.553519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:37.614652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 511/1033] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:37.881991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957344d-9d46-4c64-bd12-6e2ba1c48a19 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:37.985074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957344d-9d46-4c64-bd12-6e2ba1c48a19 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:37.987847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:38.019784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.020420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.020694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.021514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.031433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:38.036272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:38.037819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:38.039367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 523/1034] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:38.044105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.144311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065c941-3fc8-4b38-9f42-5e53a689cbad None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.178741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.179343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.179620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.179889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.180550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:38.180851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:38.182209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:38.194832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 512/1035] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:38.195975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.294638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def7c43-08e9-49c2-bc06-2e34e115720c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.327790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.328463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.328741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.329011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.338792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:38.340237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:38.355050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:38.355983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:38.363325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:38.363325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:38.363325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:38.363325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 524/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:38.363609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.377024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:38.479447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4518c-825f-401c-b644-f043ddeb4728 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.512193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.513035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.513572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.514020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.515151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:38.515643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:38.517178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:38.529410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 513/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:38.535059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.554154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:38.638313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece5654-f275-46d9-b796-8098fd87cf44 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.672493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.673108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.673381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.673645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.681557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:38.683012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:38.694010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:38.694967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:38.695920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:38.697060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 525/1038] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:38.700871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.772897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:38.805089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7cf48-9b24-4647-9d4d-4fb52595ef9d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.839025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.839619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:38.839892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:38.840160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:38.840870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:38.841169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:38.842569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:38.864558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 514/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:38.865699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:38.941960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:38.965806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937710ec-2d62-46e3-9320-b167dc71cd04 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:38.999206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:38.999832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.000077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.000342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.065841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.067259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.141398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.170806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 526/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:39.172221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.272437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844888b-1297-43cd-8b82-ae761b638859 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.301683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.311404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.312030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.312306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.312573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.323311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.324676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e9d0c082-6bf9-455a-8f8f-85856fc22e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:39.339009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:39.339944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:39.340851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:39.340851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:39.340851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:39.340851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 515/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:39.346824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.391312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535588e-4428-4e18-9584-228fe6a9a190 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.391594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535588e-4428-4e18-9584-228fe6a9a190 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.391861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535588e-4428-4e18-9584-228fe6a9a190 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.393328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 516/1042] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:39.398543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.398862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.399139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.417766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:39.419000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:39.450389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d3f2e9-87da-405a-9f5c-b2a0108a17c0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.490990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.491601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.491878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.492195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.492782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:39.493082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.494457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98207fe9-bdbe-4ac4-bb33-aaf3df7c4846 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.497914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 517/1043] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:39.510912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 527/1044] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:39.512072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.613706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c4b2b-73b7-4211-9411-01e8cdd92896 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.624843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.649024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.649655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.649929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.650199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.656789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.658220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.659904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f49eb-e382-43d3-88cd-756776457300 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.660367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f49eb-e382-43d3-88cd-756776457300 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.660781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f49eb-e382-43d3-88cd-756776457300 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.662844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 528/1045] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:39.667767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.668115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.668454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:39.669900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:39.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:39.671746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 518/1046] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:39.679256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.691867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:39.693246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:39.725197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.770442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 529/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:39.772004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.786183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e32fe-5ae8-4c4c-91d6-8ad8ec1f3b2d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.824857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.825461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.825737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.826006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.826724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:39.827026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.828443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.840908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 519/1048] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:39.846746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.876233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.879870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521de2-eb2a-42d5-a8ce-5518839c37f1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.915212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.915839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.916110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.916376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.916950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.918303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:39.926829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:39.927859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:39.928701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 530/1049] 10.222.0.20 () {56 vars in 1105 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:39.933816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:39.953785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f738b5f-4d54-487b-b1c4-fd12f0eb1879 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:39.965594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:39.991318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:39.991920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:39.992190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:39.992532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:39.993235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] RBAC: Adding query filter params (role.id=fed9876de0fd463fa75d25a2e6ce7fe7, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:39.993533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:39.994967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.010793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 520/1050] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=fed9876de0fd463fa75d25a2e6ce7fe7 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:40.043663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.051767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7755ed-3b84-4327-b0dd-1add40f4c3f4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.089422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.090051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.090331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.090632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.091273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] RBAC: Adding query filter params (name=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:40.091575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.092980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.102836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 531/1051] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:40.104595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.157698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:40.166062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca574136-49f1-40f4-81f9-5bd8303b03ae None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.205129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.205837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.206188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.206490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.209017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:40.216693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc97a-fc46-4221-97e4-3678f32fde15 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.217772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:40.222613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:40.223539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:40.224633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 521/1052] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:40.229614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.256170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.256779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.257059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.257318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.257956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] RBAC: Adding query filter params (type=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:40.258257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.259654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.265142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 532/1053] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:40.270479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.345055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe634-d063-4910-8505-b616c60a9d0e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.372105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06348e1-72d1-495e-91be-bafb7b07e276 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.382680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.383286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.383563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.383832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.384605 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:40.384903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.386384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.400636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 522/1054] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:40.401665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.410112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.410681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.410958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.411227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.412423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.413788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.422027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:40.437016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 533/1055] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:12:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:12:40.528538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd649b-2165-4bd2-a18f-d4bcadd3d924 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.572373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.573001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.573273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.573540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] PATH_INFO: `/v3/users/a87325bf8e3f4d8d85799958079ea9b5` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.589682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] RBAC: Authorizing `identity:get_user(user_id=a87325bf8e3f4d8d85799958079ea9b5)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.591175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.597767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 523/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/users/a87325bf8e3f4d8d85799958079ea9b5 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:40.599351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.617052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:40.706933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595fc73-07fb-4929-999d-e8b135554436 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:40.740899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:40.741510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:40.741784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:40.742048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:40.808489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:40.809946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:40.824797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:40.914282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 534/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:12:40.915521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:40.985476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:41.017447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921889b5-68a9-4e83-8ecb-f565638f03a6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:41.051198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:41.051805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:41.052103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:41.052364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:41.053452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:41.054895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f7d618-f636-43b2-8a08-da18bfda44ec None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:41.064634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 524/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:12:40 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:41.064798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:41.164787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75507-8a70-4ea8-a9ea-749780a25564 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:41.178959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:41.201679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:41.202292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:41.202571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:41.202978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:41.203707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=0d595728c0a3411797750499d98d38bf, user.id=a87325bf8e3f4d8d85799958079ea9b5) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:41.204015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:41.205434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9088d-187c-4b11-af8b-7bc434adcd46 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:41.214620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 535/1059] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:41 2022] GET /identity/v3/role_assignments?user.id=a87325bf8e3f4d8d85799958079ea9b5&scope.project.id=0d595728c0a3411797750499d98d38bf&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:42.156465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d1a2c-440c-4881-8fa4-f0c8606db5da None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.157020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d1a2c-440c-4881-8fa4-f0c8606db5da None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.157020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d1a2c-440c-4881-8fa4-f0c8606db5da None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.158507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 525/1060] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:42.163909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdf6d-1f55-4e0a-b55e-a41c949a5816 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.164290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdf6d-1f55-4e0a-b55e-a41c949a5816 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.164468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdf6d-1f55-4e0a-b55e-a41c949a5816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.182836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-45fbdf6d-1f55-4e0a-b55e-a41c949a5816 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:42.184019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-45fbdf6d-1f55-4e0a-b55e-a41c949a5816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:42.249289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 536/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:42.514762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d1a2c-440c-4881-8fa4-f0c8606db5da None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:42.616794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d1a2c-440c-4881-8fa4-f0c8606db5da None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:42.620523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c4ac-39f2-4a67-877c-d363b054757a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.620806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c4ac-39f2-4a67-877c-d363b054757a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.621075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392c4ac-39f2-4a67-877c-d363b054757a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.622657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 537/1062] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:12:42.627471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.627747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.628026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.647651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:42.648835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:42.652997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:42.653620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.653896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.654166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.662064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:42.667371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:42.672171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:42.673152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:42.673993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:42.675266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 526/1063] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:42.679112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:42.725433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 538/1064] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:42.782376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc24ca-1646-4a06-a6a3-2042dcda6132 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:42.817897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:42.818494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.818797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.819066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.819721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:42.820020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:42.821369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:42.829533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 527/1065] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:42.835078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:42.876175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:42.936397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c45b38-bb3d-4ef1-8212-b174c96edff1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:42.978383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:42.978979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:42.979258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:42.979522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:42.989424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:42.990893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:42.995440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:43.005538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:43.006521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:43.012318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:43.012318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.012318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:43.012318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 539/1066] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:43.012618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.063037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.110934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.111137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c00498-1da9-4160-9640-e8bd992426f6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.121442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817465-8e51-4a50-a2b0-2ba8d565a805 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.151311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.152125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.152565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.152974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.157556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.158160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.158442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.158744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.159395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:43.159699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.161085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.173584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 540/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:43.179628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.228698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.230076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.287842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63021f17-b0ca-4b64-ba4c-e90dd98576c6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.295117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.325676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.326307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.326590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.326880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.333989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.335465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.344212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 528/1068] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:43.348318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:43.349803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:43.350784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 541/1069] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:43.352581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.355868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.369324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.453516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856ab67-3897-47f5-ae8f-6534a14720a1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.454761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fec823-a875-4f2f-98a8-6fdda9577c60 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.488096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.488283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.488635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.488817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.488924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.489089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.489193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.489932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.490497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:43.490835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.492261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4516-f19d-455d-90e7-b632ff37d588 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.500091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.501592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.509055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 542/1070] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:43.509055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61809-2ff0-4514-a35d-adf0ad085d20 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.509055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61809-2ff0-4514-a35d-adf0ad085d20 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.509055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61809-2ff0-4514-a35d-adf0ad085d20 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.509055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 543/1071] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:43.515396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f61809-2ff0-4514-a35d-adf0ad085d20 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a1f6f682-ff76-44ea-a55d-ff2488d01019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:43.516303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:43.517373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:43.518320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:43.518320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.518320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:43.518320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 529/1072] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:43.518829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.519110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.519380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.537208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:43.538399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:43.545551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.580743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.604430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 530/1073] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:43.605249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.615346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f61809-2ff0-4514-a35d-adf0ad085d20 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.651345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.652082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.652409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.652715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.664097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.665607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:43.681489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:43.682666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:43.683731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:43.683731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:43.683731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:43.683731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 544/1074] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:43.688298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.707868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51c3e2-2c8e-40e0-9609-03cb56f725eb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.743840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.744447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.744795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.745069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.745704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:43.746004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.747459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.758819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 531/1075] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:43.793219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b1b8-a956-4eee-a8d5-2234e75b3a83 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.819977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.844696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:43.845421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:43.845774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:43.846114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:43.846868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:43.847241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:43.849301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:43.862843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 545/1076] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:43.868899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.871859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:43.881315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:43.986104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41bb2b9-4b5e-4bf5-996c-1027a5a33dab None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:43.998976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432e975-ea06-43b8-83bb-c582ad8f94f8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.023435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.024111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.024380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.024648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.032652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.034085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.040868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.041507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.041793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.042068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:44.048070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:44.049165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:44.050264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 532/1077] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:44.055778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.056190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:44.060942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:44.061955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:44.062850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:44.066682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 546/1078] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:12:44.068286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.124757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:44.125505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:44.166065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb627a-39f4-4efe-a808-1e2c64501765 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.182865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79c44-c69e-4784-8353-55a0ca3134b7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.198470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.199088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.199377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.199636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.200319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:44.200620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.202030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.218619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 533/1079] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:44.220138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.221431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.222037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.222310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.222594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.223273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:44.223568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.224925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.237517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 547/1080] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:44.239358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.315293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7435f-80df-48b2-82cc-bd78ff3b3a1d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.336154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674bf8-9da5-46f3-9416-0ec7ef11191f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.337559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:44.337939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:44.350299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.350971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.351248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.351517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/fed9876de0fd463fa75d25a2e6ce7fe7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.353661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=04ddfa59d1834ba3a7f1f27e963b090e, role_id=fed9876de0fd463fa75d25a2e6ce7fe7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.355114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.367713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 534/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:44 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/04ddfa59d1834ba3a7f1f27e963b090e/roles/fed9876de0fd463fa75d25a2e6ce7fe7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:12:44.370539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.371206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.371486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.371757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] PATH_INFO: `/v3/users/a87325bf8e3f4d8d85799958079ea9b5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.372940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] RBAC: Authorizing `identity:get_user(user_id=a87325bf8e3f4d8d85799958079ea9b5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.374414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.376333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 548/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:44 2022] GET /identity/v3/users/a87325bf8e3f4d8d85799958079ea9b5 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:44.382862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.493748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0accf7-fb2f-4024-8770-fef0e0bc0116 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.531540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.532218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.532518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.532792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.534007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.535474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6c7ed-b800-4b81-92ae-d5b3e7143985 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.537135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 535/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:12:44 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:44.543272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:44.642101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc844079-99ea-481d-b132-82cd39f9ad83 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:44.674449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:44.675130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:44.675402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:44.675667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf/users/a87325bf8e3f4d8d85799958079ea9b5/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:44.678103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d595728c0a3411797750499d98d38bf, user_id=a87325bf8e3f4d8d85799958079ea9b5, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:44.679552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825c29-ba12-4034-88b4-f38060567395 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:44.692240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 549/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:44 2022] PUT /identity/v3/projects/0d595728c0a3411797750499d98d38bf/users/a87325bf8e3f4d8d85799958079ea9b5/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:12:46.532018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42840-84e2-4329-a26e-70e498b60340 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:46.532592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42840-84e2-4329-a26e-70e498b60340 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:46.532592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42840-84e2-4329-a26e-70e498b60340 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:46.533872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 536/1085] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:46.539559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31301-2a74-4c79-9d9f-50d11e61efd0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:46.540012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31301-2a74-4c79-9d9f-50d11e61efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:46.540114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31301-2a74-4c79-9d9f-50d11e61efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:46.557838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b7b31301-2a74-4c79-9d9f-50d11e61efd0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:46.559055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b7b31301-2a74-4c79-9d9f-50d11e61efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:46.582062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:46.624334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 550/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:46.830257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d637359-4b30-4a44-b930-7a559abbbbc8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:46.830437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d637359-4b30-4a44-b930-7a559abbbbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:46.830735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d637359-4b30-4a44-b930-7a559abbbbc8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:46.832066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 537/1087] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:46.837778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:46.838066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:46.838340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:46.856507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:46.857689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:46.896886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d637359-4b30-4a44-b930-7a559abbbbc8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:46.923681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 551/1088] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:46.947302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:46.996289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d637359-4b30-4a44-b930-7a559abbbbc8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.028931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.029539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.029813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.030082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.039915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:47.044658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:47.045827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 18 06:12:47.046954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:47.047944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 538/1089] 10.222.0.20 () {56 vars in 1120 bytes} [Sun Sep 18 06:12:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:47.052265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.143862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.154422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73ad1c-1656-4321-85f6-5600a37096aa None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.187626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.188276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.188492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.188754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.189409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:47.189701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.191096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.197930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.199173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 552/1090] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:47.204811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.296092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9336ae-a0cf-41a8-a4ad-d57c077ee585 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.298974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2270ec4-7286-4e92-8b9e-7f150a44fa38 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.306287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.329655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.330269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.330545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.330837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.330956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.331594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.331873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.332144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.340629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:12:47.342984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.344461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:12:47.345468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:47.346446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:12:47.347609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 539/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:47.352933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:47.361589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:47.362755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:47.363939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:47.363939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:47.363939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:47.363939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 553/1092] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:47.368345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.371880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.456795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e3be6-8491-485c-a84f-4cc805484ee5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.468029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5806cf6-97e2-425b-ac3b-7f453776d5c6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.494372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.495010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.495283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.495551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.496210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:47.496513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.497861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.500178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.500771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.501046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.501310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.501932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:47.502226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.503595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.505850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 540/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:47.511629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.518622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 554/1094] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:47.521508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.542511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.580855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.612175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea29ff-56dd-490f-b93b-68caa3748c79 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.618163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de565d0-a9ee-4df2-825b-a820a99108f3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.646647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.647264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.647542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.647813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] PATH_INFO: `/v3/users/a87325bf8e3f4d8d85799958079ea9b5` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.648924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] RBAC: Authorizing `identity:get_user(user_id=a87325bf8e3f4d8d85799958079ea9b5)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.650340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.650924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.651541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.651817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.656551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 541/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/users/a87325bf8e3f4d8d85799958079ea9b5 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:47.656694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.657498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.659139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.660558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:12:47.671792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:47.677960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 555/1096] 10.222.0.20 () {56 vars in 1093 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:47.679192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.757244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.760383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5866c77-d756-4b21-899c-fb8ba8f079b9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.774767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdc7bb-1dcd-46eb-b5ac-4137ec9ec2a1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.794782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.795376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.795652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.795925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.796818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.807732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.808329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.808601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.808867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:47.809579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:47.809877 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.811468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.829321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 556/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:47.835210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.864914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:47.866534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:47.932667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae25363-b12b-4647-8eb4-4e7fdd26e0f1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:47.956714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:47.962874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 542/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:47.965303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:47.965912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:47.966014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:47.966191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:47.966461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.016951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:48.035957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.037396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.063145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68d14e-37e8-446e-a88c-f308ea5a65f4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.096143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.096750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.097023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.097290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.098384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.099839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.101603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 543/1099] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:48.107474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.134019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 557/1100] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:12:48.139963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.172670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:48.207290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1aa1a0-5c84-45a3-9d38-4d545839202f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.217483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:48.235732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491087a-c2c9-4ac4-abd2-a9c8d918419d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.241693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.242291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.242569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.242868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.243589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=0d595728c0a3411797750499d98d38bf, user.id=a87325bf8e3f4d8d85799958079ea9b5) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:48.243890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.245315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.251341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 544/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/role_assignments?user.id=a87325bf8e3f4d8d85799958079ea9b5&scope.project.id=0d595728c0a3411797750499d98d38bf&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:12:48.270953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.271521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.271797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.272063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.283330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.284816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:12:48.300591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:48.302297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:12:48.303463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:12:48.303463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:12:48.303463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:48.306784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 558/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:48.308371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.411930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1f4-bc12-4958-af5a-6de764c8e4c7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.443372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:48.448865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.449472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.449750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.450019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.450689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:48.450989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.452489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.464092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 545/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:48.469671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.567222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305281c3-6ba4-4eff-b129-d9037b081f96 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.600695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.601309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.601592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.601864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.603741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:48.609715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.611254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:12:48.622345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:12:48.623479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:12:48.624423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 559/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:12:48.629385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.726627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5b93d-8e6a-400d-92e9-75f6f00ecfd1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.759338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.759936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.760211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.760478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.761197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:48.761497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.762934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854f441-c8a6-410d-87e1-e2a010be5c92 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.775522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 546/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:48.781353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:48.878407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37ce9-882d-48a8-b7cc-b35744255ebe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:48.911929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:48.912512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:48.912791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:48.913059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:48.913774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] RBAC: Adding query filter params (role.id=fed9876de0fd463fa75d25a2e6ce7fe7, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=04ddfa59d1834ba3a7f1f27e963b090e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:48.914080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:48.915527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c22b0-e803-4f86-904d-61dc0c107a36 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:48.921205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 560/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:48 2022] GET /identity/v3/role_assignments?user.id=04ddfa59d1834ba3a7f1f27e963b090e&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=fed9876de0fd463fa75d25a2e6ce7fe7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:50.406967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be902d3-ba51-45d6-94ef-fa639a74b712 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:50.407499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be902d3-ba51-45d6-94ef-fa639a74b712 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:50.407499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be902d3-ba51-45d6-94ef-fa639a74b712 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:50.408837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 547/1107] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:50.414410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:50.414862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:50.414956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:50.433129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:50.434301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:50.485178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:50.498521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 561/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:50.767927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be902d3-ba51-45d6-94ef-fa639a74b712 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:50.820153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:50.866676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be902d3-ba51-45d6-94ef-fa639a74b712 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:50.899835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:50.900441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:50.900718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:50.900991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:50.902088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:50.903533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef5306-4cf3-4984-a109-ded3d80a1ddf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:50.905080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 548/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:12:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:50.910685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:51.002434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0673f57-08a9-460c-b683-3938fe4b4065 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:51.034570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:51.035134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:51.035407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:51.035671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:51.036832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:51.038239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6fb64-7db8-4a97-9333-7e60c9c0c3a7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:51.058698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 562/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:12:50 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:12:53.195648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787deb2a-381f-4948-a9f0-be8de19171b5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:53.196182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787deb2a-381f-4948-a9f0-be8de19171b5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:53.196182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787deb2a-381f-4948-a9f0-be8de19171b5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:53.197519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 549/1111] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:53.202708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b194ea-c123-4979-86c4-195418454f3d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:53.203142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b194ea-c123-4979-86c4-195418454f3d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:53.203243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b194ea-c123-4979-86c4-195418454f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:53.221368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-55b194ea-c123-4979-86c4-195418454f3d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:53.222545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-55b194ea-c123-4979-86c4-195418454f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:53.260481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:53.287166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 563/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:12:53.550045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787deb2a-381f-4948-a9f0-be8de19171b5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:53.648349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787deb2a-381f-4948-a9f0-be8de19171b5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:53.667968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:53.681615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:53.682233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:53.682509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:53.682810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] PATH_INFO: `/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:53.684183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] RBAC: Authorizing `identity:get_role(role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:53.685550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:53.686897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 550/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:53 2022] GET /identity/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:53.692762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b194ea-c123-4979-86c4-195418454f3d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:53.787213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b194ea-c123-4979-86c4-195418454f3d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:53.818908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:53.819521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:53.819797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:53.820066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] PATH_INFO: `/v3/users/cba7a5be97dc4d358fb47086e7f0abd4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:53.832963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] RBAC: Authorizing `identity:get_user(user_id=cba7a5be97dc4d358fb47086e7f0abd4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:53.834402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:53.835995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 564/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:53 2022] GET /identity/v3/users/cba7a5be97dc4d358fb47086e7f0abd4 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:53.841532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:53.940131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7d20-2568-4313-b6e5-c186ba089cef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:53.973408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:53.973995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:53.974274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:53.974546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:54.028341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:54.041061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:54.042434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:54.136384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 551/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:12:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:12:54.142460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:54.172363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:54.237874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc44370-274d-4712-a5d3-9205a67cd160 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:54.272177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:54.272795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:54.273067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:54.273333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:54.274407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:54.276010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f81c74-75dd-49b5-ad97-44ea08d997dc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:54.277607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 565/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:12:54 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:54.283724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:54.367019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:54.378545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4e9d5-bee0-42cc-a782-56c99edefbfd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:54.411283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:54.411864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:54.412138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:54.412406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:54.413115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0d595728c0a3411797750499d98d38bf, user.id=cba7a5be97dc4d358fb47086e7f0abd4) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:12:54.413419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:54.414851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfcea5-1d46-4bc4-9532-afef8313bd77 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:54.419955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 552/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:12:54 2022] GET /identity/v3/role_assignments?user.id=cba7a5be97dc4d358fb47086e7f0abd4&scope.project.id=0d595728c0a3411797750499d98d38bf&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:56.507511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f090ca-9cba-43d7-8634-e34005df781c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:56.508010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f090ca-9cba-43d7-8634-e34005df781c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:56.508101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f090ca-9cba-43d7-8634-e34005df781c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:56.509394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 566/1118] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:56.515110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:56.515542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:56.515650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:56.533150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:56.534319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:56.596664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 553/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:56.863542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f090ca-9cba-43d7-8634-e34005df781c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:56.956555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:56.961257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f090ca-9cba-43d7-8634-e34005df781c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:56.992734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:56.993342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:56.993618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:56.993890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] PATH_INFO: `/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:56.995299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] RBAC: Authorizing `identity:get_role(role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:56.996654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:56.997941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 567/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:56 2022] GET /identity/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:57.003576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:57.080500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:57.098510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e89c26-4004-465e-9720-75ddc779b8b5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:57.130796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:57.131352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:57.131627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:57.131892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] PATH_INFO: `/v3/users/cba7a5be97dc4d358fb47086e7f0abd4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:57.132984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] RBAC: Authorizing `identity:get_user(user_id=cba7a5be97dc4d358fb47086e7f0abd4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:57.134389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:57.135967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 554/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:57 2022] GET /identity/v3/users/cba7a5be97dc4d358fb47086e7f0abd4 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:57.141849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:57.236906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b9b2a-d990-4b0b-b3ab-ed6c8f66277d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:57.269224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:57.270354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:57.270660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:57.270933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:57.272634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:57.275332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bcc786-b3ba-46b4-bd56-9a8b5572f991 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:57.277014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 568/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:12:57 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:12:57.283508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:57.378227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684139b-f15d-4633-818d-960d41bb9e72 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:12:57.411411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:12:57.412041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:57.412310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:57.412571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf/users/cba7a5be97dc4d358fb47086e7f0abd4/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:57.414680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d595728c0a3411797750499d98d38bf, user_id=cba7a5be97dc4d358fb47086e7f0abd4, role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:12:57.416079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df082a8c-f18a-451f-96f0-652d15b12197 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:12:57.428191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 555/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:12:57 2022] PUT /identity/v3/projects/0d595728c0a3411797750499d98d38bf/users/cba7a5be97dc4d358fb47086e7f0abd4/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:12:59.525985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b436-2cbf-4bcc-8a7b-1407557bb928 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:59.526490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b436-2cbf-4bcc-8a7b-1407557bb928 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:59.526608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8b436-2cbf-4bcc-8a7b-1407557bb928 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:59.527920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 569/1124] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:12:59.537835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:12:59.538263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:12:59.538416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:12:59.556392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:12:59.557569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:12:59.560250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:59.621800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 556/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:12:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:12:59.891655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b436-2cbf-4bcc-8a7b-1407557bb928 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:12:59.926731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:12:59.988905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8b436-2cbf-4bcc-8a7b-1407557bb928 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:00.021638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:00.022252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:00.022528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:00.022823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] PATH_INFO: `/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:00.024193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] RBAC: Authorizing `identity:get_role(role_id=6a79de4a0cf14d0187f2f6f7637a1287)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:00.025550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:00.026963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 570/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:12:59 2022] GET /identity/v3/roles/6a79de4a0cf14d0187f2f6f7637a1287 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:00.032739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:00.126256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c1a9f-7124-411c-b0d7-c1a6095f1a2e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:00.159063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:00.159676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:00.159951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:00.160245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] PATH_INFO: `/v3/users/cba7a5be97dc4d358fb47086e7f0abd4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:00.161341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] RBAC: Authorizing `identity:get_user(user_id=cba7a5be97dc4d358fb47086e7f0abd4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:00.162781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:00.164368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 557/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:00 2022] GET /identity/v3/users/cba7a5be97dc4d358fb47086e7f0abd4 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:00.169995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:00.267515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579f00b-4b4b-4c11-bc95-807cfd1435c9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:00.269731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:00.299268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:00.299871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:00.300150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:00.300421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:00.363964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:00.365400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:00.458434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 571/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:13:00.464182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:00.475281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:00.565170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb384d-058b-42a8-b50c-5837531509eb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:00.597621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:00.598881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:00.599159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:00.599428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] PATH_INFO: `/v3/projects/0d595728c0a3411797750499d98d38bf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:00.600533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] RBAC: Authorizing `identity:get_project(project_id=0d595728c0a3411797750499d98d38bf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:00.601949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7c14-7afb-47e2-adc8-6ce61e373def None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:00.603601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 558/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:00 2022] GET /identity/v3/projects/0d595728c0a3411797750499d98d38bf => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:00.609750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:00.612863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:00.704272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7a38d-cb8f-439d-b69d-4445759a8200 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:00.737234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:00.737779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:00.738066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:00.738342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:00.739140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] RBAC: Adding query filter params (role.id=6a79de4a0cf14d0187f2f6f7637a1287, scope.project.id=0d595728c0a3411797750499d98d38bf, user.id=cba7a5be97dc4d358fb47086e7f0abd4) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:00.739450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:00.740885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc278c3-a18c-4187-9f40-6ca9b643c0b5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:00.746622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 572/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:13:00 2022] GET /identity/v3/role_assignments?user.id=cba7a5be97dc4d358fb47086e7f0abd4&scope.project.id=0d595728c0a3411797750499d98d38bf&role.id=6a79de4a0cf14d0187f2f6f7637a1287 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:02.880130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd38b8-eeb4-4b69-bfa4-9f6732e08343 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:02.880641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd38b8-eeb4-4b69-bfa4-9f6732e08343 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:02.880641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd38b8-eeb4-4b69-bfa4-9f6732e08343 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:02.881904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 559/1131] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:02.887296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:02.887700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:02.887843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:02.906123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:02.907342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:02.965410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:02.971132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 573/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:03.236465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd38b8-eeb4-4b69-bfa4-9f6732e08343 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:03.312229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:03.335290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd38b8-eeb4-4b69-bfa4-9f6732e08343 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:03.367353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:03.367959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:03.368234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:03.368503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:03.369594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:03.371046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3802f8d-2846-44b8-a5d4-875716275bc4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:03.372584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 560/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:13:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:03.378154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:03.473089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee39e-3c0b-409f-89f4-a318a5654a45 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:03.506449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:03.507015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:03.507296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:03.507568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:03.508227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:03.509652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24571ea3-4c7d-4cd5-aa81-4f971bc75014 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:03.535062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 574/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:13:03 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:13:05.650607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be7e6-aee4-4ae4-825f-1eb12183bcfa None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:05.651131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be7e6-aee4-4ae4-825f-1eb12183bcfa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:05.651131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92be7e6-aee4-4ae4-825f-1eb12183bcfa None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:05.652487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 561/1135] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:05.657763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:05.658183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:05.658304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:05.676441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:05.677605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:05.720523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:05.741041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 575/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:06.006224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92be7e6-aee4-4ae4-825f-1eb12183bcfa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:06.104655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92be7e6-aee4-4ae4-825f-1eb12183bcfa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:06.137659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:06.138265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:06.138540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:06.139096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:06.140191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:06.141610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3436f-cd07-4c3f-9a7e-782e5f18ebbb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:06.143192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 562/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:13:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:06.149486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:06.243007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81bbac-4194-47d2-8c62-1143e5a56deb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:06.275424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:06.275960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:06.276239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:06.276510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:06.277147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:06.278567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b9e2a-9cb2-4585-bf50-4176387e9902 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:06.298775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 576/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:13:06 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:13:08.427825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20d31-8f11-4c0c-ba94-2c9a1b322fd1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:08.428381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20d31-8f11-4c0c-ba94-2c9a1b322fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:08.428381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb20d31-8f11-4c0c-ba94-2c9a1b322fd1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:08.429696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 563/1139] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:08.434899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:08.435301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:08.435446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:08.453996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:08.455211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:08.483644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:08.521920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 577/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:08.786246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb20d31-8f11-4c0c-ba94-2c9a1b322fd1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:08.801133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:08.885020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb20d31-8f11-4c0c-ba94-2c9a1b322fd1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:08.917758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:08.918368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:08.918675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:08.918950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:08.929228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:08.934007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:08.935091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:08.936213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 564/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:08.941104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:09.036424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebdfe0-a3a0-4833-9c43-150923b0fba6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:09.068901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:09.069508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:09.069764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:09.070031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:09.070722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:09.071020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:09.072374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:09.079962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 578/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:09.085856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:09.154086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:09.182939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd894b-a3de-4ac0-ad6a-3afcad2e3136 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:09.214772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:09.215370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:09.215644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:09.215911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] PATH_INFO: `/v3/users/cb003406861745e5b215504a2c95f7a2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:09.228617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] RBAC: Authorizing `identity:get_user(user_id=cb003406861745e5b215504a2c95f7a2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:09.230032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:09.231604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 565/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:09 2022] GET /identity/v3/users/cb003406861745e5b215504a2c95f7a2 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:09.237274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:09.264851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:09.332961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771e21-138f-4bbb-927d-04ac9bc7ac28 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:09.364632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:09.365222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:09.365497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:09.365765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:09.428780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:09.430210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:09.457141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:09.526188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 579/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:13:09.532026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:09.626386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1541627-f97e-4ec1-a137-fa2685359601 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:09.653941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:09.659891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:09.660505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:09.660780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:09.661050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] PATH_INFO: `/v3/projects/77419a4675b94923aecd63e9ca2e5c67` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:09.662142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] RBAC: Authorizing `identity:get_project(project_id=77419a4675b94923aecd63e9ca2e5c67)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:09.663591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207db13-86f8-4d73-aed9-c3eddea5f32c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:09.665227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 566/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:09 2022] GET /identity/v3/projects/77419a4675b94923aecd63e9ca2e5c67 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:09.671094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:09.764461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fdd1d1-a128-4ee1-9aef-a668bc14b32b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:09.789584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:09.797253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:09.797861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:09.798136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:09.798403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:09.799155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=77419a4675b94923aecd63e9ca2e5c67, user.id=cb003406861745e5b215504a2c95f7a2) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:09.799458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:09.801470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717c5e0-d624-45f2-8730-4cc4970d2ad2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:09.806650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 580/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:13:09 2022] GET /identity/v3/role_assignments?user.id=cb003406861745e5b215504a2c95f7a2&scope.project.id=77419a4675b94923aecd63e9ca2e5c67&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:11.883792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498eead-5e10-4cfb-a657-9c5be7ce26e3 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:11.884295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498eead-5e10-4cfb-a657-9c5be7ce26e3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:11.884295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498eead-5e10-4cfb-a657-9c5be7ce26e3 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:11.885576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 567/1147] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:11.891146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:11.891553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:11.891682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:11.909118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:11.910897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:11.973487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 581/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:12.245253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498eead-5e10-4cfb-a657-9c5be7ce26e3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:12.342936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498eead-5e10-4cfb-a657-9c5be7ce26e3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:12.374789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:12.375389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:12.375659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:12.375921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:12.385491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:12.390448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:12.391556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:12.392709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 568/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:12.397912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:12.492787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd81775-6957-43c0-aa86-164886e7033d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:12.514087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:12.525873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:12.526474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:12.526786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:12.527059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:12.527715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:12.528013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:12.529375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:12.537814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 582/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:12.543543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:12.600017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:12.640139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a5bd2-cee5-4914-80b1-3036955b1c7d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:12.674121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:12.674765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:12.675043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:12.675330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] PATH_INFO: `/v3/users/cb003406861745e5b215504a2c95f7a2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:12.676441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cb003406861745e5b215504a2c95f7a2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:12.677867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:12.679471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 569/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:12 2022] GET /identity/v3/users/cb003406861745e5b215504a2c95f7a2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:12.685244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:12.781379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b48a42-6eeb-473a-824a-ee28fde75b00 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:12.812783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:12.813393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:12.813670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:12.813937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] PATH_INFO: `/v3/projects/77419a4675b94923aecd63e9ca2e5c67` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:12.815055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] RBAC: Authorizing `identity:get_project(project_id=77419a4675b94923aecd63e9ca2e5c67)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:12.817026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a5c7b-7e79-4b19-a542-2bd7cb78782c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:12.818626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 583/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:12 2022] GET /identity/v3/projects/77419a4675b94923aecd63e9ca2e5c67 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:12.824250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:12.918133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd17c-8476-4a95-a452-b324d34784a2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:12.942532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:12.958805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:12.959736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:12.960013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:12.960281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] PATH_INFO: `/v3/projects/77419a4675b94923aecd63e9ca2e5c67/users/cb003406861745e5b215504a2c95f7a2/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:12.962901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] RBAC: Authorizing `identity:create_grant(project_id=77419a4675b94923aecd63e9ca2e5c67, user_id=cb003406861745e5b215504a2c95f7a2, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:12.964676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8dcab-9a3d-41b0-ad67-bb460befa612 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:12.978612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 570/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:13:12 2022] PUT /identity/v3/projects/77419a4675b94923aecd63e9ca2e5c67/users/cb003406861745e5b215504a2c95f7a2/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:13:15.062940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f6dc-3a26-406f-af51-a1c43fba9cef None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:15.063474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f6dc-3a26-406f-af51-a1c43fba9cef None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:15.063474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407f6dc-3a26-406f-af51-a1c43fba9cef None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:15.064833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 584/1154] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:15.070068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:15.070500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:15.070637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:15.088873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:15.090045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:15.154960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 571/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:15.438875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407f6dc-3a26-406f-af51-a1c43fba9cef None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:15.491058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:15.537465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407f6dc-3a26-406f-af51-a1c43fba9cef None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:15.569327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:15.569952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:15.570228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:15.570498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:15.580691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:15.586132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:15.587261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:13:15.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 585/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:15.593881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:15.690489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65696-9327-4863-b2ca-90685ec9fe07 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:15.706891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:15.725131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:15.725721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:15.725996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:15.726265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:15.726951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:15.727251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:15.728602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:15.736277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 572/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:15.742177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:15.838534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442508ff-6aff-4da1-ab70-0a8608876869 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:15.850961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:15.871672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:15.872287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:15.872564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:15.872834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] PATH_INFO: `/v3/users/cb003406861745e5b215504a2c95f7a2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:15.873920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] RBAC: Authorizing `identity:get_user(user_id=cb003406861745e5b215504a2c95f7a2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:15.875390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:15.876955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 586/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:15 2022] GET /identity/v3/users/cb003406861745e5b215504a2c95f7a2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:15.883318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:15.985188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6170e-50d4-4493-b05e-bbeaa558df59 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:16.018435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:16.019092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:16.019369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:16.019639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:16.062996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:16.084930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:16.086363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:16.179500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 573/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:13:16.185382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:16.287654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf994f08-3c10-46d0-921d-b42dc073dbc8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:16.318471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:16.319114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:16.319387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:16.319652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] PATH_INFO: `/v3/projects/77419a4675b94923aecd63e9ca2e5c67` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:16.320742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] RBAC: Authorizing `identity:get_project(project_id=77419a4675b94923aecd63e9ca2e5c67)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:16.322161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cdd49-d99a-41b6-9fba-a1a66820ce88 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:16.323803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 587/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:16 2022] GET /identity/v3/projects/77419a4675b94923aecd63e9ca2e5c67 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:16.329893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:16.404728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:16.425419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8240c8d-6ded-44cf-88a6-1b310d9258db None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:16.458264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:16.458884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:16.459156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:16.459422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:16.460137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=77419a4675b94923aecd63e9ca2e5c67, user.id=cb003406861745e5b215504a2c95f7a2) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:16.460433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:16.461906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f4b421-28c1-4ea5-96f7-a9821abc59ae None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:16.467583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 574/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:13:16 2022] GET /identity/v3/role_assignments?user.id=cb003406861745e5b215504a2c95f7a2&scope.project.id=77419a4675b94923aecd63e9ca2e5c67&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:18.616628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870eed-949c-46f2-9a66-77fe6f9b17bf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:18.617166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870eed-949c-46f2-9a66-77fe6f9b17bf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:18.617166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870eed-949c-46f2-9a66-77fe6f9b17bf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:18.618535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 588/1162] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:18.623693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:18.624127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:18.624230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:18.642744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:18.644512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:18.707581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 575/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:18.973846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870eed-949c-46f2-9a66-77fe6f9b17bf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:19.022816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:19.072714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870eed-949c-46f2-9a66-77fe6f9b17bf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:19.105544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:19.106137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:19.106414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:19.106708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:19.169701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:19.171179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:19.215815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:19.264459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 589/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:13:19.270813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:19.336674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:19.365018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704f7df-3662-42c3-8b21-f0472bbe36b6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:19.397549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:19.398158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:19.398433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:19.398733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:19.405124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:19.406551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a62d1100-88e8-40c3-adcb-86c4f70e853a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:19.418120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:19.419454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:19.420456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 576/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:13:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:19.425569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:19.519763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a251097-fd68-4e08-8022-9838cb8c4ac2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:19.552293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:19.552885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:19.553159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:19.553423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:19.554046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:19.554341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:19.555718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd86bc1-92bc-4482-8c35-7c54be2e9b7e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:19.562488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 590/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Sun Sep 18 06:13:19 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:21.677581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a1da-23be-4a9d-a60c-72c98ce8693e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:21.678106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a1da-23be-4a9d-a60c-72c98ce8693e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:21.678106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a1da-23be-4a9d-a60c-72c98ce8693e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:21.679467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 577/1167] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:21.684722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053744f-284a-4d8c-9472-4ade34c162bd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:21.685105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053744f-284a-4d8c-9472-4ade34c162bd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:21.685282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053744f-284a-4d8c-9472-4ade34c162bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:21.703399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e053744f-284a-4d8c-9472-4ade34c162bd None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:21.704583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e053744f-284a-4d8c-9472-4ade34c162bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:21.712315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:21.770571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 591/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:22.036286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a1da-23be-4a9d-a60c-72c98ce8693e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:22.135539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a1da-23be-4a9d-a60c-72c98ce8693e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:22.162467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:22.167685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:22.168272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:22.168546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:22.168813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:22.169477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:22.170891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845c984-480f-4664-8057-b1b9fa8070b7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:22.186017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 578/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Sun Sep 18 06:13:22 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:13:24.306983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363cc4-fa1d-47d3-b3bc-80d4b9fed970 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:24.307537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363cc4-fa1d-47d3-b3bc-80d4b9fed970 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:24.307537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98363cc4-fa1d-47d3-b3bc-80d4b9fed970 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:24.308867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 592/1170] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:24.314032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:24.314398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:24.314585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:24.332785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:24.333950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:24.397449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 579/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:24.660678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98363cc4-fa1d-47d3-b3bc-80d4b9fed970 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:24.757813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98363cc4-fa1d-47d3-b3bc-80d4b9fed970 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:24.789944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:24.790558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:24.790869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:24.791141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] PATH_INFO: `/v3/domains/3941ba13d1f5499ab090105c060a2d93` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:24.801538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=3941ba13d1f5499ab090105c060a2d93)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:24.803010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7f729-7020-44a1-b56c-81d98949b6ec None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:24.804566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 593/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Sun Sep 18 06:13:24 2022] GET /identity/v3/domains/3941ba13d1f5499ab090105c060a2d93 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:24.810379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:24.902917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a382665-484f-4d63-b129-cfb727426c87 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:24.924966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:24.936889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:24.937483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:24.937781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:24.938050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:24.938737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:24.940200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4414b42d-f36f-4d71-8db3-b85b98757c4f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:24.965756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 580/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:13:24 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:13:27.125995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15be87-3775-4a88-a367-9ceee4ac23a6 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:27.126532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15be87-3775-4a88-a367-9ceee4ac23a6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:27.126532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15be87-3775-4a88-a367-9ceee4ac23a6 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:27.128443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 594/1174] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:27.133663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:27.134169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:27.134271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:27.153033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:27.154209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:27.216713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 581/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:13:27.484774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15be87-3775-4a88-a367-9ceee4ac23a6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:27.487743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:27.583875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15be87-3775-4a88-a367-9ceee4ac23a6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:27.615956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:27.616572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:27.616850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:27.617121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] PATH_INFO: `/v3/domains/3941ba13d1f5499ab090105c060a2d93` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:27.618203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=3941ba13d1f5499ab090105c060a2d93)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:27.619687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bcae5-8c11-45e5-a6b2-0e3ed4d71c5d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:27.621224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 595/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Sun Sep 18 06:13:27 2022] GET /identity/v3/domains/3941ba13d1f5499ab090105c060a2d93 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:27.626811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:27.721867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ee275-1859-4d7d-b703-e2963c529f08 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:27.724033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:27.755129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:27.755710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:27.755986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:27.756251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:27.756893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:27.758300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6c506-6243-4ffd-bd21-2d0f80d4a1cc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:27.778691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 582/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:13:27 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:13:29.923075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06fd2-1ffd-472e-9d03-4f9891e5dcaf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:29.923613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06fd2-1ffd-472e-9d03-4f9891e5dcaf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:29.923613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06fd2-1ffd-472e-9d03-4f9891e5dcaf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:29.925012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 596/1178] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:29.930065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:29.930476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:29.930611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:29.948804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:29.949973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:30.013736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 583/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:30.280135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a06fd2-1ffd-472e-9d03-4f9891e5dcaf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:30.348540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:30.379936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a06fd2-1ffd-472e-9d03-4f9891e5dcaf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:30.411846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:30.412454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:30.412728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:30.412992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:30.422481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:30.427665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:30.428766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:13:30.429721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 597/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:30.434886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:30.520331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:30.530763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcad684-1079-4c1e-9e99-c821ee64c511 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:30.562680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:30.563271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:30.563544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:30.563810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:30.564459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:30.565330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:30.566702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:30.574294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 584/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:30.580147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:30.681926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdd7f2-9695-484f-93aa-56cc4783f1e4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:30.708770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:30.714758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:30.715377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:30.715653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:30.715920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] PATH_INFO: `/v3/users/10601bfa649c464ebf2f2620dfd882a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:30.728861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] RBAC: Authorizing `identity:get_user(user_id=10601bfa649c464ebf2f2620dfd882a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:30.730297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:30.731914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 598/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:30 2022] GET /identity/v3/users/10601bfa649c464ebf2f2620dfd882a1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:30.738531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:30.833552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73457-6806-4597-a047-57ff869fd190 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:30.865908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:30.866490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:30.866796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:30.867069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:30.890000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:30.935735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:30.937175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:31.039902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 585/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:13:31.045806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:31.139273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7fd1-6eb7-48fa-81c6-3b0e64d358c1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:31.171907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:31.172523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:31.172802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:31.173073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] PATH_INFO: `/v3/projects/b59c18dff2c34ade8c737d41146efd29` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:31.174694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] RBAC: Authorizing `identity:get_project(project_id=b59c18dff2c34ade8c737d41146efd29)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:31.176129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32916ab-c6cf-4ef1-8f14-f604a84bfb58 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:31.177734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 599/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:31 2022] GET /identity/v3/projects/b59c18dff2c34ade8c737d41146efd29 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:31.183642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:31.234837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:31.281336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1ae6a-cf81-4bff-8c73-df14d1b85782 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:31.321999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:31.322615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:31.322896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:31.323165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:31.323884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=b59c18dff2c34ade8c737d41146efd29, user.id=10601bfa649c464ebf2f2620dfd882a1) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:31.324190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:31.325599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70064f00-fdd5-46f2-8426-44b8f0f1cbc4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:31.330970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 586/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:13:31 2022] GET /identity/v3/role_assignments?user.id=10601bfa649c464ebf2f2620dfd882a1&scope.project.id=b59c18dff2c34ade8c737d41146efd29&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:13:33.452070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84ab9-28bb-4fda-9eb1-7b469ed6560a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:33.452630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84ab9-28bb-4fda-9eb1-7b469ed6560a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:33.452630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e84ab9-28bb-4fda-9eb1-7b469ed6560a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:33.453965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 600/1186] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:33.459251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:33.459644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:33.459788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:33.477499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:33.478720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:33.541878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 587/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:33.811869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84ab9-28bb-4fda-9eb1-7b469ed6560a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:33.867015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:33.910588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e84ab9-28bb-4fda-9eb1-7b469ed6560a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:33.943491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:33.944100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:33.944380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:33.944652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:33.954330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:33.959684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:33.960760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:13:33.961612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 601/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:33.966677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:34.010725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:34.060227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6164c-90bb-49e3-abcd-f29905611ea9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:34.092146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:34.092678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:34.092948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:34.093210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:34.093855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:34.094148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:34.095513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:34.103122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 588/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:34.108760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:34.208371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e54f2c-76fb-4b9d-a1f8-e820904bbce2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:34.224043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:34.242010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:34.242665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:34.242947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:34.243218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] PATH_INFO: `/v3/users/10601bfa649c464ebf2f2620dfd882a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:34.244325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] RBAC: Authorizing `identity:get_user(user_id=10601bfa649c464ebf2f2620dfd882a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:34.245748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:34.247366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 602/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:34 2022] GET /identity/v3/users/10601bfa649c464ebf2f2620dfd882a1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:34.253794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:34.348970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fbbf1-3c79-4a1e-b8d3-7beeee1c6e11 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:34.357239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:34.381131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:34.381739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:34.382009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:34.382274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] PATH_INFO: `/v3/projects/b59c18dff2c34ade8c737d41146efd29` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:34.383395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] RBAC: Authorizing `identity:get_project(project_id=b59c18dff2c34ade8c737d41146efd29)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:34.384799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74279d-cace-41f4-ab71-05f209939292 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:34.386398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 589/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:34 2022] GET /identity/v3/projects/b59c18dff2c34ade8c737d41146efd29 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:34.392611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:34.488580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768300e9-66c6-42e7-a2cb-0af479607c45 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:34.520252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:34.520892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:34.521165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:34.521432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] PATH_INFO: `/v3/projects/b59c18dff2c34ade8c737d41146efd29/users/10601bfa649c464ebf2f2620dfd882a1/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:34.523272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] RBAC: Authorizing `identity:create_grant(project_id=b59c18dff2c34ade8c737d41146efd29, user_id=10601bfa649c464ebf2f2620dfd882a1, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:34.524682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413eeaa-c307-4709-b328-44a168f6fe3e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:34.537344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 603/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:13:34 2022] PUT /identity/v3/projects/b59c18dff2c34ade8c737d41146efd29/users/10601bfa649c464ebf2f2620dfd882a1/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:13:36.632790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcfde-f929-4050-80a8-976e8fb77d05 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:36.633305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcfde-f929-4050-80a8-976e8fb77d05 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:36.633394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bcfde-f929-4050-80a8-976e8fb77d05 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:36.634687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 590/1193] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:13:36.640241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:36.640657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:36.640765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:36.660125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:13:36.661930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:13:36.699850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:36.727158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 604/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:13:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:13:36.999205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bcfde-f929-4050-80a8-976e8fb77d05 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.095312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bcfde-f929-4050-80a8-976e8fb77d05 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:37.127235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:37.127833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:37.128104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:37.128370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:37.138600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:13:37.143747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:13:37.144767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:13:37.146005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 591/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:13:37.150666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.243590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a458-94ca-4fda-8825-787bf77caf50 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:37.275939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c978-e801-4138-a922-44643f915bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:37.276466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c978-e801-4138-a922-44643f915bef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:37.276738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c978-e801-4138-a922-44643f915bef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:37.277003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c978-e801-4138-a922-44643f915bef None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:37.277652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c978-e801-4138-a922-44643f915bef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:37.277948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c978-e801-4138-a922-44643f915bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:37.279334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c978-e801-4138-a922-44643f915bef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:37.287504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 605/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:37.293226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.326719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:37.388886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154a32-ae4e-459f-a860-d03fff634d82 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:37.422415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:37.423131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:37.423409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:37.423709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] PATH_INFO: `/v3/users/10601bfa649c464ebf2f2620dfd882a1` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:37.424816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] RBAC: Authorizing `identity:get_user(user_id=10601bfa649c464ebf2f2620dfd882a1)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:37.426236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:37.432510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 592/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Sun Sep 18 06:13:37 2022] GET /identity/v3/users/10601bfa649c464ebf2f2620dfd882a1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:37.433632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c978-e801-4138-a922-44643f915bef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.475244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:37.542457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c978-e801-4138-a922-44643f915bef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:37.575130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:37.575736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:37.576016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:37.576285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:37.643789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:37.645367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:37.678234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:37.739300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 606/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:13:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:13:37.745256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.841320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb2a5a-f47d-4b24-83b2-78b4753dfcce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:37.843458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:37.873079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:37.873680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:37.873950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:37.874214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] PATH_INFO: `/v3/projects/b59c18dff2c34ade8c737d41146efd29` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:37.875326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] RBAC: Authorizing `identity:get_project(project_id=b59c18dff2c34ade8c737d41146efd29)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:37.876734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64764f-af6c-4609-86d2-e2636ae80e01 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:37.878321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 593/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Sun Sep 18 06:13:37 2022] GET /identity/v3/projects/b59c18dff2c34ade8c737d41146efd29 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:37.884266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:13:37.977822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4c6a4-2e76-45a7-a1ae-2760e3674c40 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:13:38.009379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:13:38.009963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:13:38.010233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:13:38.010497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:13:38.011233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=b59c18dff2c34ade8c737d41146efd29, user.id=10601bfa649c464ebf2f2620dfd882a1) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:13:38.011534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:13:38.012940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2a0a9-01a8-4a29-b78c-cb775d943954 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:13:38.015902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:13:38.020666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 607/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:13:37 2022] GET /identity/v3/role_assignments?user.id=10601bfa649c464ebf2f2620dfd882a1&scope.project.id=b59c18dff2c34ade8c737d41146efd29&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:13:47.869177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=14 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:33.097923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0924e5-e8c1-448b-ad35-d924db5dc080 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:33.098731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0924e5-e8c1-448b-ad35-d924db5dc080 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:33.098731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0924e5-e8c1-448b-ad35-d924db5dc080 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:33.099889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 594/1201] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:14:33.106056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:33.106816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:33.106816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:33.126098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:33.127305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:33.209747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 608/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:14:33.502454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0924e5-e8c1-448b-ad35-d924db5dc080 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:33.631376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0924e5-e8c1-448b-ad35-d924db5dc080 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:33.682050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:33.687503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:33.687503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:33.687503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:33.698211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:33.699689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:33.715221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:33.716294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:33.717342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:33.717342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:33.717342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:33.721462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 595/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:33.722441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:33.843477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283e508-6ee5-4f4b-8d2c-18ff90f6a199 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:33.888447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:33.894876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:33.895516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:33.895814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:33.896183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:33.896990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:33.897336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:33.899126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89db4ff-6a1d-4c61-a8ad-3c2f2b95bfad None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:33.917761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 609/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:33.925402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:33.994767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:34.059594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e079bc0-e350-47c3-b6f4-18cbeef55807 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:34.101588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:34.102489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:34.102821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:34.103215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:34.104113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:34.106010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1e257-bdce-4487-a979-9027851b3cce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:34.146643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 596/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Sun Sep 18 06:14:33 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:14:36.443396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135908f-0c1f-411a-976e-2b5d9697112a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:36.443396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135908f-0c1f-411a-976e-2b5d9697112a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:36.444493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135908f-0c1f-411a-976e-2b5d9697112a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:36.446178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 610/1206] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:14:36.454614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:36.454614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:36.454614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:36.472378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:36.473596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:36.546069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 597/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:14:36.821209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135908f-0c1f-411a-976e-2b5d9697112a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:36.950510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135908f-0c1f-411a-976e-2b5d9697112a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:36.993209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:36.994423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:36.994808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:36.995079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:37.012731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:14:37.018079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:37.019431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:37.020275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 611/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:37.027109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:37.081829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:37.158710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fb397-3bf3-464e-9ae1-7f6f454fa2e5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:37.197577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:37.199029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:37.199029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:37.199029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:37.200399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:37.200399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:37.202148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:37.212081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 598/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:14:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:37.218950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:37.280212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:37.339923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ff4e7-78ed-48ec-919d-d1ff32dba8b8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:37.388801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:37.389520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:37.389855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:37.390191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:37.403319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:37.405631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:37.424522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:37.426119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:37.427139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:37.427139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:37.427139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:37.427139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 612/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:37.432273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:37.572995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:37.576796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973db642-9bc1-419c-baf8-cb0339357ff0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:37.617732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:37.618087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:37.618422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:37.619008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:37.619904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:37.620307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:37.621927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:37.638608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 599/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:37.643193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:37.800765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b6cb2-411f-4371-990b-306af4d39ebb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:37.802927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:37.837851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:37.839945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:37.839945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:37.839945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:37.849041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:37.850702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:14:37.863303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:37.865287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:37.866685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 613/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:37.871224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:38.012038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2dc3a-00f7-4507-8f3a-b0b7cfef8f74 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:38.081919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:38.082902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:38.083131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:38.083430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:38.084164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:38.084503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:38.085891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:38.112375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 600/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:14:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:38.113596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:38.224262 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bab42-1bdf-461b-97af-5a36fda24389 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:38.263054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:38.263784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:38.264180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:38.264515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:38.295567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:38.339181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:38.340812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:38.450366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 614/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:14:38.456307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:38.476325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:38.567548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605177-e8e6-41a3-83d3-bd51023340e2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:38.605447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:38.606864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:38.608006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:38.608277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:38.618983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:38.620496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:38.640542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:38.641789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:38.642727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:38.642727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:38.642727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:38.649899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 601/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:38.650062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:38.713786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:38.766606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5b275-30ee-4be5-ad51-41399f53d731 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:38.801807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:38.803218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:38.803218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:38.803218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:38.804111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:38.804541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:38.806021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:38.827799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 615/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:38.829060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:38.916994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:38.955468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ee6c-f8e2-4f7c-a68e-786e9f280730 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:38.990813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:38.991478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:38.991793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:38.992066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:39.002742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:39.004187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:14:39.017217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:39.018500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:39.019765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:39.021275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 602/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:39.027039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:39.152323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891f24-2339-4df8-b162-f7ee5da93238 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:39.184769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:39.199536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:39.200232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:39.200542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:39.201467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:39.202288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:39.202648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:39.204085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9d8f4-6807-4b69-ade6-81b830df89b0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:39.217874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 616/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:39.224138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:39.341370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa013a2-0f27-400e-9191-66b3a679bc68 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:39.373355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:39.381385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:39.381985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:39.382258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:39.382559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:39.383350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=cc74075ebcdd4916aacec44e87e86064) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:39.383638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:39.385423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba03ea37-8b39-4362-ad1f-e2f80fb5aa7f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:39.392377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 603/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:14:39 2022] GET /identity/v3/role_assignments?user.id=cc74075ebcdd4916aacec44e87e86064&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:41.597592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640163-049b-4d53-b67f-521bbccc50da None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:41.598152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640163-049b-4d53-b67f-521bbccc50da None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:41.598152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640163-049b-4d53-b67f-521bbccc50da None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:41.599665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 617/1220] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:14:41.605293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:41.605778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:41.605884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:41.624587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:41.625672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:41.693212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 604/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:14:41.978282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640163-049b-4d53-b67f-521bbccc50da None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.089802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640163-049b-4d53-b67f-521bbccc50da None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:42.127937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:42.128588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:42.128941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:42.129129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:42.143726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:14:42.149893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:42.151267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:42.152165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 618/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:14:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:42.157739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.265655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748e89a-cf7a-437d-8824-d11adf40a010 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:42.291966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:42.304429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:42.305034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:42.305312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:42.305585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:42.306250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:42.306624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:42.308115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:42.317676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 605/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:14:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:42.323524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.377202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:42.443170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef1354-4fcf-4b55-8a81-ea28fbf27cc2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:42.479535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:42.480165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:42.480440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:42.480708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:42.490628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:42.492122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:42.507608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:42.514172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:42.515382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:42.515382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:42.515382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:42.515382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 619/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:42.515712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.636490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8cf08-b5d9-4a3c-830b-e6f149f57c9e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:42.682414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:42.683173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:42.683485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:42.683792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:42.684489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:42.684829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:42.686227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:42.701429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 606/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:42.707414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.828229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:42.835440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ccfff-29fa-41a2-956d-dbc7e5ec1ef0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:42.872985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:42.873703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:42.873990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:42.874303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:42.884040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:42.885572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:14:42.900029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:42.901206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:42.902727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 620/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:42.907881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:42.964505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:43.031991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff70b4-3c13-4901-b00b-63ec0c69cc36 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.066177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.066799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.067071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.067364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.068050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:43.068390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.069960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.087606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 607/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:14:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:43.094014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:43.200066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ea79-cc74-41e3-92ac-1e659303302c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.233114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.233732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.234005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.234269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.244513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.245955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:43.260211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:43.261470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:43.262380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:43.262380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:43.262380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:43.262380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 621/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 18 06:14:43.268117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:43.374140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d031e-25da-4156-8d95-bc03e3b62f37 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.379146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:43.422013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.422723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.423118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.423651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.424654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:43.424962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.426764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.440186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 608/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:43.446456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:43.459532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:43.568756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178706b-84c1-471e-88ac-2f6d0096c1ce None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.603084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.603710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.603984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.604252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.610813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.612281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:14:43.624196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:43.625281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:43.626359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 622/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:43.631485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:43.738771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71615-21a8-427e-839f-0b27bfbead91 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.774809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.775531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.775810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.776083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.776774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:43.777073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.778498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c08cc-268b-49c1-afc7-5ff1fb48808f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.790597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 609/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:43.797410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:43.864499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:43.900159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f139c-d227-4c74-ae61-9a5688dc2d75 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:43.938341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:43.939118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:43.939312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:43.939583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/cc74075ebcdd4916aacec44e87e86064/roles/201a3f3f8b7348eb94b4834b8104f279` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:43.954712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=cc74075ebcdd4916aacec44e87e86064, role_id=201a3f3f8b7348eb94b4834b8104f279)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:43.956443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953fc39-69e1-4485-aab2-c0e96c81dab6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:43.968640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 623/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:14:43 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/cc74075ebcdd4916aacec44e87e86064/roles/201a3f3f8b7348eb94b4834b8104f279 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:14:46.196941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0fbd-2ba4-4849-b62e-4962b3ecb7fd None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:46.197482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0fbd-2ba4-4849-b62e-4962b3ecb7fd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:46.197482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0fbd-2ba4-4849-b62e-4962b3ecb7fd None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:46.198862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 610/1233] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:14:46.204306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:46.204733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:46.204855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:46.225103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:46.226344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:46.301753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 624/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:14:46.642524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0fbd-2ba4-4849-b62e-4962b3ecb7fd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:46.674321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:46.772311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0fbd-2ba4-4849-b62e-4962b3ecb7fd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:46.814285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:46.815187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:46.815460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:46.815729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:46.827524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:14:46.833390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:46.834903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:46.835954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 611/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Sun Sep 18 06:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:46.841113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:46.892291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:46.951111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f83cf-4ffa-468e-a029-c46ed15d5f9b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:46.985213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:46.985829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:46.986112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:46.986379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:46.987076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:46.987375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:46.988738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:46.996728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 625/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Sun Sep 18 06:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:47.003346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:47.098352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:47.115463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bf3c6-cffb-4727-8e96-c47ed511041e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:47.164967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:47.165972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:47.165972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:47.165972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:47.177479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:47.178939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:47.195654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:47.196850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:47.198190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:47.198190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:47.198190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:47.198190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 612/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:47.202870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:47.310481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49330fad-4c73-42fd-bc33-962ba634b9f0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:47.320174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:47.348309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:47.349121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:47.349455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:47.349823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:47.350669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:47.351061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:47.352707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:47.365049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 626/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:47.370756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:47.477479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdfdfa-5b75-4c55-9f88-936012c39dd6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:47.513786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:47.514410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:47.514716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:47.514982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:47.517713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:47.523804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:47.525658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:14:47.539441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:47.540590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:47.541651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 613/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:47.548038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:47.664581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59529e2-dd4c-4213-bf78-10ca6a580394 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:47.709063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:47.709806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:47.710278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:47.710494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:47.711276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:47.711628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:47.713311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:47.731464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 627/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:14:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:14:47.739429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:47.854567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ceddd-e7f6-493a-a2a0-43edaf7e0d09 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:47.894139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:47.894765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:47.895037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:47.895302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:47.960906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:47.971319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:47.972806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.092739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 614/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:14:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 18 06:14:48.098962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:48.132749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:48.218443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3477503-ea52-499a-99b8-301dc4b8026e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:48.258694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:48.259433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:48.259730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:48.259999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:48.272206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:48.273850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:48.291479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:48.292774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:48.294463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:48.294463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:48.294463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:48.294463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 628/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:48.299108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:48.401938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:48.410137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12a07f-7735-4ecf-b78b-6958e4c51d99 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:48.446556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:48.447305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:48.447522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:48.447856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:48.448567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:48.448913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:48.450941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.463614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 615/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:14:48.470557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:48.557683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:48.585154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9138-85dd-428c-998d-642639bd07e6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:48.620149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:48.620781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:48.621058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:48.621325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:48.628616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:48.630114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:14:48.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:48.645292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:48.646275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 629/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:48.651585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:48.763018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a294-3f16-407c-ae2e-4b5baa99d830 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:48.802989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:48.803589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:48.803866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:48.804135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:48.804819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:48.805123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:48.806731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776888b-c6fd-4617-a94c-1de31beb3bd3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.819683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 616/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:48.826094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:48.937274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81ccf0-51c6-42b4-9d97-4d5a1917f791 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:48.975577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:48.976476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:48.976784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:48.977080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:48.977847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] RBAC: Adding query filter params (role.id=201a3f3f8b7348eb94b4834b8104f279, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=cc74075ebcdd4916aacec44e87e86064) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:48.978190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:48.979819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e93d52-bd2d-41bc-b173-ef36ae705e41 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:48.986450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 630/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:14:48 2022] GET /identity/v3/role_assignments?user.id=cc74075ebcdd4916aacec44e87e86064&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=201a3f3f8b7348eb94b4834b8104f279 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:51.620491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824d67-e7a1-48fd-8712-85b86ae5820f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:51.621038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824d67-e7a1-48fd-8712-85b86ae5820f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:51.621038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96824d67-e7a1-48fd-8712-85b86ae5820f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:51.622548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 617/1247] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:14:51.627814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:51.628342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:51.628342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:51.635557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:51.650795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:51.652097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:51.730047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 631/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:14:51.996913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824d67-e7a1-48fd-8712-85b86ae5820f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.016493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:52.111577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96824d67-e7a1-48fd-8712-85b86ae5820f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.148587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.149475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.149475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.149857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.161105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:14:52.166376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:52.167661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:52.168496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 618/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:52.174478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.280992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa0d63-133b-4539-9091-54ebeecba444 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.310102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:52.315614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.316225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.316494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.316761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.317419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:52.317712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:52.319132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:52.327927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 632/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:52.344469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.421422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:52.444655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bc9f7-cb1f-4912-a16c-57a10cc94466 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.484438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.485075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.485353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.485633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.497619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:52.499146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:52.515329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:52.516376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:52.517678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:52.517678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:52.517678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:52.517678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 619/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:52.522515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.626447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e231f7-2b55-4b8c-aef8-d59a49ae83a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.661564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.662258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.662447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.662745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.663402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:52.663704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:52.665110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:52.678652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 633/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:52.684848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.791164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c407e-79ce-4ba5-887f-713a2bab4fbe None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.816883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:52.825295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.825930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.826206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.826476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.834063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:52.835519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:14:52.847155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:52.848291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:52.849293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 620/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:52.854234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:52.917428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:52.962858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b7ef0-2803-4728-a369-15b258a8d89a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:52.996278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:52.996910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:52.997188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:52.997454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:52.998180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:52.998478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:52.999927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.017344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 634/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:14:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:14:53.023510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.118281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dacdb-05a8-4b9f-b92e-bbfea7f0415a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:53.161417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:53.162237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:53.162645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:53.163100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:53.219073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:53.231299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:53.232729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.325805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 621/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:14:53.331538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.424710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee83c-e4cd-444f-95f2-e46d017f56d6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:53.437773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:53.458487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:53.459140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:53.459415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:53.459681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:53.469590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:53.470984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:53.485587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:53.486717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:53.487624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:53.487624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:53.487624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:53.487624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 635/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:53.492429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.576224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:53.588258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd0897-48d5-4e42-a92c-1c2761c85d6f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:53.620482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:53.621062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:53.621334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:53.621597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:53.622824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:53.623119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:53.624463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.635029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 622/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:53.640795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.734080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f212b-5e22-4a8d-b7b7-b0cfa07f4c15 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:53.765604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:53.766203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:53.766475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:53.766767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:53.772971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:53.774393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:14:53.785126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:53.786243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:14:53.787384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 636/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:53.791826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.887196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec820a3b-7430-4822-b437-820175b4981c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:53.919061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:53.919641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:53.919914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:53.920182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:53.920853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:53.921149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:53.922543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe5efd-b67e-4388-80c8-53988d79ece4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:53.933897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 623/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:53.940345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:53.965246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:54.036055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf02afa-45c1-4535-87f2-a629bb4c0d6d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:54.067644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:54.068221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:54.068493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:54.068757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:54.069462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=cc74075ebcdd4916aacec44e87e86064) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:54.069759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:54.071191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38bd1f-7785-4a4b-b73e-024e175586eb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:54.076871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 637/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:14:53 2022] GET /identity/v3/role_assignments?user.id=cc74075ebcdd4916aacec44e87e86064&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:14:56.172008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e43b2-5cb8-480e-80e3-8ec7ddfdf47d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:56.172582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e43b2-5cb8-480e-80e3-8ec7ddfdf47d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:56.172774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e43b2-5cb8-480e-80e3-8ec7ddfdf47d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:56.174900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 624/1261] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:14:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:14:56.180442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:56.180898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:56.181022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:56.202112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:14:56.203344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:14:56.267121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 638/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:14:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:14:56.539852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e43b2-5cb8-480e-80e3-8ec7ddfdf47d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:56.550548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:56.636741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e43b2-5cb8-480e-80e3-8ec7ddfdf47d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:56.669669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:56.670278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:56.670557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:56.670861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:56.680619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:14:56.685396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:56.686486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:56.687377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 625/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:56.692464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:56.710470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:56.787712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332ee77-5d45-40c3-877a-874d5a0210e7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:56.819750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:56.820347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:56.820621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:56.820887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:56.821540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:56.821836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:56.823230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:56.831451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 639/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:56.837263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:56.904698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:56.937456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff923e-b860-422b-af38-63eda5b3a37a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:56.969943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:56.970551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:56.970857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:56.971123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:56.980686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:56.982744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:56.996742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:56.997831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:56.998773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:56.998773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:56.998773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:56.998773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 626/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:57.003690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.080504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:57.100327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992188b-4cdf-4aa3-83e4-ea7ceffd4d3d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.133885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74222-0def-4275-b394-7966908841ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.134474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74222-0def-4275-b394-7966908841ac None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.134800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74222-0def-4275-b394-7966908841ac None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.135053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74222-0def-4275-b394-7966908841ac None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.135688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74222-0def-4275-b394-7966908841ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:57.136232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74222-0def-4275-b394-7966908841ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.138235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74222-0def-4275-b394-7966908841ac None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.149785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 640/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:57.156464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.253011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff08b5-b7d7-4a12-8c92-fbf082ec235d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.271701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:57.285418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.286013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.286286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.286550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.294157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.295587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:14:57.306277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:57.307366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:57.308521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 627/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:57.313695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74222-0def-4275-b394-7966908841ac None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.408316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74222-0def-4275-b394-7966908841ac None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.441330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.441925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.442206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.442479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.443234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:57.443542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.444958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.453358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:57.462145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 641/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:57.467909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.563830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992aa26-8df5-4301-9950-f4df50621ed0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.596067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.596672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.596947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.597215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.607894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.609347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:14:57.622891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:57.623977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:14:57.625200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:14:57.625200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:14:57.625200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:57.625200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 628/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:57.629655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.724398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b7200-ae2a-494c-af39-3af4cc756a0c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.756568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.757158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.757437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.757702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.758328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:57.758649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.760002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.771207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 642/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:57.776619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.807728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:57.873874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae192f-8927-44dd-9ae9-d67542b1025c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:57.906084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:57.906704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:57.906980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:57.907245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:57.913585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:57.915044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:14:57.926442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:14:57.927502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:14:57.928625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 629/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:14:57.933192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:57.989776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:58.030202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a931c-387b-4f21-a584-6b3d86ea1185 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:58.061129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:58.061715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:58.061989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:58.062254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:58.062954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:14:58.063251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:58.064655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e1eee-d0d0-4906-9195-0a17b767dcb4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:58.076387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 643/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:14:58.082304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:14:58.178286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:14:58.187790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf3f50-9af8-4a81-b667-3425189ab711 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:14:58.218975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:14:58.219608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:14:58.219879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:14:58.220145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] PATH_INFO: `/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/cc74075ebcdd4916aacec44e87e86064/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:14:58.221974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] RBAC: Authorizing `identity:create_grant(project_id=602a93d2c7f749dea6f26019f01dfa9c, user_id=cc74075ebcdd4916aacec44e87e86064, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:14:58.223412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1481f-d608-49db-8e60-f51b59beeacb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:14:58.235838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 630/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Sun Sep 18 06:14:58 2022] PUT /identity/v3/projects/602a93d2c7f749dea6f26019f01dfa9c/users/cc74075ebcdd4916aacec44e87e86064/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:15:00.335533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3896-444a-4d40-a2c7-e302b1950471 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:00.336098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3896-444a-4d40-a2c7-e302b1950471 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:00.336098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a3896-444a-4d40-a2c7-e302b1950471 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:00.337391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 644/1274] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:15:00.343150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:00.343537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:00.343696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:00.361551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:15:00.362879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:15:00.437236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 631/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:15:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:15:00.710023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a3896-444a-4d40-a2c7-e302b1950471 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:00.807700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a3896-444a-4d40-a2c7-e302b1950471 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:00.825930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:00.843547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:00.844179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:00.844456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:00.844728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:00.854618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 18 06:15:00.859387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:00.860478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:00.861364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 645/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Sun Sep 18 06:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:00.866785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:00.945603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:00.964377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831a20a-8f76-4137-985f-a85b7e35bf9c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:00.997804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:00.998406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:00.998715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:00.998989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:00.999651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:00.999950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.001386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.010008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 632/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Sun Sep 18 06:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:01.016156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:01.115709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2327d-9778-4725-9309-69e2fc3ac023 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:01.154246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:01.155050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:01.155358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:01.155745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:01.166545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.169031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:15:01.189050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:01.191268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:15:01.192593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:15:01.192593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:15:01.192593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:01.192593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 646/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:01.197548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:01.294268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a297a-2698-4629-8406-2eb3bdfc7a3d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:01.326123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:01.326739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:01.327016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:01.327285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:01.327917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:01.328215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.329562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.341353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 633/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:01.347088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:01.377229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:01.444132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae5bb3-9f72-4c8b-b689-dc628a933b78 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:01.484286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:01.484925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:01.485204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:01.485481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:01.494838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.496318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 18 06:15:01.507301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:01.508408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:01.509639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 647/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:01.514566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:01.518384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:01.611439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c1065-9349-4a35-a878-d43922f79505 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:01.643032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:01.643615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:01.643887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:01.644153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:01.644857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:01.645154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.646549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.663105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 634/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:01.669462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:01.756360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:01.766912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a5924-57d7-442e-9d7e-e1a13d672560 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:01.799083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:01.799672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:01.799948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:01.800215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:01.865487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:01.866882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:01.951438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:01.961605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 648/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 18 06:15:01.968011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:02.026017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:02.063088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81f95d-f928-4c51-b14a-cb8f0dd80cbd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:02.095901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:02.096506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:02.096779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:02.097046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:02.107005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:02.108415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 18 06:15:02.123019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:02.124056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 18 06:15:02.125002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 18 06:15:02.125002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 18 06:15:02.125002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:15:02.125002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 635/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:02.130270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:02.226285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c884-f82d-46a9-b0f6-ed1dc3c99381 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:02.258643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:02.259237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:02.259520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:02.259792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:02.260425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:02.260992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:02.262957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:02.273600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 649/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Sun Sep 18 06:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:02.279366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:02.374799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecd143-ab4e-4454-9d8f-2ca7ea8b614b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:02.388296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:02.407477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:02.408077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:02.408349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:02.408615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:02.414922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:02.416333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 18 06:15:02.427049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:02.428073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Sep 18 06:15:02.429156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 636/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Sun Sep 18 06:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:02.434387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:02.482546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:02.529819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109738c0-16de-41fa-9c4b-b7e7757b88ff None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:02.561882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:02.562475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:02.562772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:02.563036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:02.563706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:02.564002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:02.565412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d1f5a-d4c1-49a8-816d-66490ba46bde None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:02.576754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 650/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Sun Sep 18 06:15:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:15:02.582800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:02.678043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780bdfe-16ed-4949-b31e-94ce5aced719 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:02.710067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:02.710676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:02.710950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:02.711213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:02.711917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] RBAC: Adding query filter params (role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2, scope.project.id=602a93d2c7f749dea6f26019f01dfa9c, user.id=cc74075ebcdd4916aacec44e87e86064) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:02.712215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:02.713610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0e1b-52ae-4da6-87a9-cfe365c8367b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:02.719126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 637/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Sun Sep 18 06:15:02 2022] GET /identity/v3/role_assignments?user.id=cc74075ebcdd4916aacec44e87e86064&scope.project.id=602a93d2c7f749dea6f26019f01dfa9c&role.id=e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:04.880935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c515-584f-488c-9ff1-7a9af481844e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:04.881452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c515-584f-488c-9ff1-7a9af481844e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:04.881452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c515-584f-488c-9ff1-7a9af481844e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:04.882753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 651/1288] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:15:04.888640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:04.889061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:04.889186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:04.907041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:15:04.908233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:15:04.936613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:04.977301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 638/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:15:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:15:05.250258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654c515-584f-488c-9ff1-7a9af481844e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:05.346762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654c515-584f-488c-9ff1-7a9af481844e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:05.357305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:05.380261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:05.380874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:05.381147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:05.381411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:05.382003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:05.383393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:05.392029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:05.393231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:05.394434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 652/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:15:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:05.401211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:05.495006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940d508-bb60-4086-9554-b0a3e8647d1a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:05.526862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:05.527902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:05.528171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:05.528440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:05.529049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] RBAC: Adding query filter params (name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:05.529345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:05.530704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981cd05-e319-4ac8-9b62-55dd28d483ea None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:05.535280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 639/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Sun Sep 18 06:15:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:05.540951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:05.638016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fb52f-928b-445b-8a01-784094bc45e3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:05.670812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:05.671407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:05.671687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:05.671962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:05.672581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:05.672883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:05.674248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda3743-4c16-4c86-b7de-d90f82e30184 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:05.678496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 653/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Sun Sep 18 06:15:05 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:07.750680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79313-4b58-4106-a34b-dc395910fd3b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:07.751179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79313-4b58-4106-a34b-dc395910fd3b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:07.751179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de79313-4b58-4106-a34b-dc395910fd3b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:07.752471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 640/1293] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:15:07.757615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7101198-b27b-4ef0-b8aa-cf1df66be2a2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:07.758023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7101198-b27b-4ef0-b8aa-cf1df66be2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:07.758152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7101198-b27b-4ef0-b8aa-cf1df66be2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:07.776599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b7101198-b27b-4ef0-b8aa-cf1df66be2a2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:15:07.777789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b7101198-b27b-4ef0-b8aa-cf1df66be2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:15:07.805520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:07.841570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 654/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:15:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:15:08.109757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de79313-4b58-4106-a34b-dc395910fd3b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:08.140696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:08.207078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de79313-4b58-4106-a34b-dc395910fd3b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:08.238871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:08.239455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:08.239729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:08.239995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:08.240637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:08.241977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb73fe4-b5a3-42c9-8a07-88bfa31a8c05 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:08.255284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 641/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Sun Sep 18 06:15:08 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:15:10.346987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9773-ce70-48e5-8fed-32463a12f299 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:10.347545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9773-ce70-48e5-8fed-32463a12f299 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:10.347545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b9773-ce70-48e5-8fed-32463a12f299 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:10.348856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 655/1296] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:15:10.354006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:10.354423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:10.354546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:10.372879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:15:10.374060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:15:10.443777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 642/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:15:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 18 06:15:10.710863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b9773-ce70-48e5-8fed-32463a12f299 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:10.807319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b9773-ce70-48e5-8fed-32463a12f299 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:10.838805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:10.839931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:10.840204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:10.840468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:10.841036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:10.842398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:10.850785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:10.851912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:10.853077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 656/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:15:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:10.858321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:10.876213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:10.955237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea6624-9705-4bbb-8957-3754c5805d7e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:10.988264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:10.988789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:10.989065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:10.989331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:10.989954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:10.990250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:10.991634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:10.996619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 643/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Sun Sep 18 06:15:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:11.002506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:11.033321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:11.097864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc2d6f-5d1d-4157-93e2-bbd42bb94e3a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:11.130666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:11.131316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:11.131687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:11.131918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:11.132706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] RBAC: Adding query filter params (interface=public, service_id=63a6ea585bcd4562a4f4a467c285e549, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:11.133059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:11.134673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd00e3-2607-492f-901b-53cc7ad87a96 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:11.140890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 657/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Sun Sep 18 06:15:11 2022] GET /identity/v3/endpoints?service_id=63a6ea585bcd4562a4f4a467c285e549&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:11.147870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:11.234476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:11.251894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d18f5-680a-467f-9327-ebedea5af7d3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:11.284940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:11.285532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:11.285807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:11.286075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:11.286699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:11.288050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059e29b-92b5-41a3-bed7-6970105091c2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:11.294672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 644/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Sun Sep 18 06:15:11 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:15:13.457057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c9ad-02d6-49d6-ba29-bca3d72ee008 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:13.457580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c9ad-02d6-49d6-ba29-bca3d72ee008 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:13.457580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073c9ad-02d6-49d6-ba29-bca3d72ee008 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:13.458955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 658/1302] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:15:13.464343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:13.464738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:13.464887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:13.483060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:15:13.484237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:15:13.547317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 645/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:15:13 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 18 06:15:13.814329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073c9ad-02d6-49d6-ba29-bca3d72ee008 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:13.904346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:13.910991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073c9ad-02d6-49d6-ba29-bca3d72ee008 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:13.942770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:13.943378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:13.943652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:13.943917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:13.944480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:13.945825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:15:13.954263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:15:13.955443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:15:13.956308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 659/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Sun Sep 18 06:15:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 18 06:15:13.961188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:14.034443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:14.057234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16240b4-4137-4b20-bf6f-78db2d4cd7ad None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:14.089436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:14.089992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:14.090268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:14.090536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:14.091185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:15:14.091484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:14.092867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32aa52-e40f-477b-897c-8b5c4a3d4343 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:14.097757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 646/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Sun Sep 18 06:15:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:15:14.103377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:15:14.204674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3eaa8-4846-4ce6-9ebd-4915745872f7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:15:14.236989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:15:14.237582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:15:14.237858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:15:14.238126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:15:14.238815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:15:14.240176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c770837-11b8-4cd2-be24-5578e30aafc7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:15:14.258557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 660/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:15:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:15:24.094878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=30 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:15:24.254383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:17.712971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b700-7d1b-40c9-af54-88a35893d005 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:17.714326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b700-7d1b-40c9-af54-88a35893d005 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:17.715113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4b700-7d1b-40c9-af54-88a35893d005 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:17.718694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 647/1307] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:17.723708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb0c-c387-4e6e-adcc-53e20dad09a9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:17.724126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb0c-c387-4e6e-adcc-53e20dad09a9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:17.724397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb0c-c387-4e6e-adcc-53e20dad09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:17.817540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c1d0fb0c-c387-4e6e-adcc-53e20dad09a9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:17.819220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c1d0fb0c-c387-4e6e-adcc-53e20dad09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:17.934480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 661/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:18.231590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:18.231872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:18.232421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:18.233859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 648/1309] 10.222.0.20 () {54 vars in 963 bytes} [Sun Sep 18 06:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:18.240314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56f95-7ec5-457d-acda-686766a8f558 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:18.241211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56f95-7ec5-457d-acda-686766a8f558 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:18.241552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab56f95-7ec5-457d-acda-686766a8f558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:18.326506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2ab56f95-7ec5-457d-acda-686766a8f558 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:18.327778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2ab56f95-7ec5-457d-acda-686766a8f558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:18.349462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:18.411241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 662/1310] 10.222.0.20 () {58 vars in 1060 bytes} [Sun Sep 18 06:16:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:18.420376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:18.422203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:18.549275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc45f5d-9d69-4697-a5c9-fee433a460f2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:18.632434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:18.633216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:18.633499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:18.633771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:18.635998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:18.708964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:18.768294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:18.769798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2778bf9-4503-4a87-aed8-d0e1185fcc14 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:18.790196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 649/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:21.243697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:21.244253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:21.244253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:21.245484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 663/1312] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:21.250569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be5b7-38d0-47d4-89fb-7f6be65c0460 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:21.250986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be5b7-38d0-47d4-89fb-7f6be65c0460 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:21.251140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910be5b7-38d0-47d4-89fb-7f6be65c0460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:21.339550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-910be5b7-38d0-47d4-89fb-7f6be65c0460 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:21.340747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-910be5b7-38d0-47d4-89fb-7f6be65c0460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:21.435480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 650/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:21.726949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:21.728552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:21.833872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ef8a8-e1fb-4fff-bdf7-c97f3eda4203 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:21.904145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:21.906422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:21.907218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:21.907665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:21.908178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:21.909790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:22.025261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:22.026638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776db73-6638-4642-91ab-808c69b8edb6 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:22.044957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 664/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:24.500240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:24.500728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:24.500836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:24.502159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 651/1315] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:24.508319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5aff-621b-4eb7-bcc7-ddbc80dbf64c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:24.508727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5aff-621b-4eb7-bcc7-ddbc80dbf64c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:24.508863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b5aff-621b-4eb7-bcc7-ddbc80dbf64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:24.572444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-403b5aff-621b-4eb7-bcc7-ddbc80dbf64c None None] MFA Rules not processed for user `3b6af059a40d42aa92232a0f849bc581`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:24.573662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-403b5aff-621b-4eb7-bcc7-ddbc80dbf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:24.623387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:24.666855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 665/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:24.954401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:24.956014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:24.966679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:25.059142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed904-d6b7-4136-9445-54b417b6d027 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:25.131011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:25.131633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:25.131870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:25.132133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:25.134012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:25.193307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:25.239390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:25.240756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead868be-24ca-41fd-b54e-ed9f4823570d service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:25.261305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 652/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:27.904078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:27.904632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:27.904632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:27.906018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 666/1318] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:27.911189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c475d-51c6-4972-9207-e88cb0d51e7a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:27.911571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c475d-51c6-4972-9207-e88cb0d51e7a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:27.912255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c475d-51c6-4972-9207-e88cb0d51e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:27.976723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4a3c475d-51c6-4972-9207-e88cb0d51e7a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:27.977911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a3c475d-51c6-4972-9207-e88cb0d51e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:28.070800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 653/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:28.348608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:28.350796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:28.449879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d37b5-5e6b-4082-a818-f4ee1e855767 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:28.523217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:28.523825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:28.524098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:28.524360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:28.525942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:28.532594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:28.630203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:28.631617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e624cc-ae30-491c-99e8-29b49eee7d46 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:28.649173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 667/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:31.407134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d0142-327a-43b4-a99d-85565b291450 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:31.407665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d0142-327a-43b4-a99d-85565b291450 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:31.407665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d0142-327a-43b4-a99d-85565b291450 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:31.409013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 654/1321] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:31.414162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22f46-9506-4e93-bc01-81a96a48109a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:31.414564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22f46-9506-4e93-bc01-81a96a48109a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:31.414767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22f46-9506-4e93-bc01-81a96a48109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:31.487653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-27c22f46-9506-4e93-bc01-81a96a48109a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:31.488887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-27c22f46-9506-4e93-bc01-81a96a48109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:31.552125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:31.594042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 668/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:31.884623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d0142-327a-43b4-a99d-85565b291450 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:31.886258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-776d0142-327a-43b4-a99d-85565b291450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:31.906965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:31.990812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d0142-327a-43b4-a99d-85565b291450 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:32.065146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:32.065746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:32.066019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:32.066284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:32.067936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:32.140167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:32.178226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:32.179733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071cdb2-79f1-4bf4-a587-2a938ada104e service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:32.198051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 655/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:34.422675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:34.423217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:34.423217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:34.425562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 669/1324] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:34.434492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022011f-8819-428a-b72e-751197690296 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:34.435071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022011f-8819-428a-b72e-751197690296 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:34.435071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022011f-8819-428a-b72e-751197690296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:34.504486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f022011f-8819-428a-b72e-751197690296 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:34.505660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f022011f-8819-428a-b72e-751197690296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:34.603816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 656/1325] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:16:34.891530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:34.893149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:34.997261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee34a00-e5e6-437a-a28d-6733687d5907 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:35.072091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:35.072698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:35.073053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:35.073320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:35.074938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:35.091916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:35.184114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:35.185562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76bdd7-253e-49c5-a70d-9ce92a0bb30b service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:35.204109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 670/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:37.411898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:37.412385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:37.412501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:37.413919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 657/1327] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:37.419294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c31f1-6dcc-427c-b26f-7f83790ce182 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:37.419701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c31f1-6dcc-427c-b26f-7f83790ce182 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:37.419833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c31f1-6dcc-427c-b26f-7f83790ce182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:37.485791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9b5c31f1-6dcc-427c-b26f-7f83790ce182 None None] MFA Rules not processed for user `3b6af059a40d42aa92232a0f849bc581`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:37.487046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b5c31f1-6dcc-427c-b26f-7f83790ce182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:37.550614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:37.582386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 671/1328] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:37.864610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:37.866158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:37.898893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:37.970494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722869c8-791d-4ba2-bbde-d83dba96263f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:38.043849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:38.044583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:38.044954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:38.045224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:38.046866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:38.124606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:38.156187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:38.157622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943713-d63b-40b6-aaee-6fa31180240a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:38.175853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 658/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:40.525633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:40.526164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:40.526164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:40.527540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 672/1330] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:40.532784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352406aa-ed89-4046-9eaa-265d3dbb357c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:40.533200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352406aa-ed89-4046-9eaa-265d3dbb357c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:40.533318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352406aa-ed89-4046-9eaa-265d3dbb357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:40.597321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-352406aa-ed89-4046-9eaa-265d3dbb357c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:40.599042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-352406aa-ed89-4046-9eaa-265d3dbb357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:40.695060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 659/1331] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:40.975131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:40.976708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:41.076560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348f1ea-d5ea-4b7d-b8f6-a505ee4f229d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:41.150089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:41.150716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:41.151001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:41.151270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:41.152880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:41.179812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:41.259504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:41.260874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8ed4b-194a-44b4-bcc6-60cd4ce28fda service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:41.278220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 673/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:43.449149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:43.449706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:43.449706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:43.451084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 660/1333] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:43.456477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3450-74fa-4871-b840-850255e820ca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:43.456902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3450-74fa-4871-b840-850255e820ca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:43.457020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e3450-74fa-4871-b840-850255e820ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:43.522257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5e7e3450-74fa-4871-b840-850255e820ca None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:43.523509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e7e3450-74fa-4871-b840-850255e820ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:43.595720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:43.617522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 674/1334] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:43.904157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:43.905717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:43.945407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:44.007564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef61020-cf78-4a4f-b1dd-2f6c7bb9af04 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:44.078909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:44.079499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:44.079774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:44.080042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:44.081621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:44.183884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:44.209973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:44.211431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8481f8-e833-46a5-916d-a6f3274b5294 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:44.229391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 661/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:46.669308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:46.669842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:46.669842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:46.671380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 675/1336] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:46.677045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b784-7f62-44fe-8d5f-2597fbdc8fc6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:46.677470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b784-7f62-44fe-8d5f-2597fbdc8fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:46.677581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525b784-7f62-44fe-8d5f-2597fbdc8fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:46.742671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4525b784-7f62-44fe-8d5f-2597fbdc8fc6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:46.743814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4525b784-7f62-44fe-8d5f-2597fbdc8fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:46.837687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 662/1337] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:47.119327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:47.120874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:47.228284 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba0350-73b3-435b-be89-15bce073a2c9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:47.298383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:47.299006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:47.299283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:47.299551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:47.301181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:47.336258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:47.407091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:47.408522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb09aa9-5446-4739-bdc9-f61ce8fcc06a service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:47.426937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 676/1338] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:51.517032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:51.517519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:51.517519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:51.523010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 663/1339] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:51.524792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd941-c2c8-44ac-ada6-6415678bbfc9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:51.525258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd941-c2c8-44ac-ada6-6415678bbfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:51.525438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bd941-c2c8-44ac-ada6-6415678bbfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:51.594105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f48bd941-c2c8-44ac-ada6-6415678bbfc9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:51.595421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f48bd941-c2c8-44ac-ada6-6415678bbfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:51.687847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:51.700271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 677/1340] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:51.982008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:51.983692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:52.032586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:52.088180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69755f-7f54-4b63-a435-9e2af2131287 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:52.170085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:52.170718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:52.170997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:52.171285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:52.172885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:52.267054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:52.283003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:52.284435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3098d6-6fb7-4bd5-8692-a1b76f9733b1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:52.303227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 664/1341] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:16:55.130069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:55.130674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:55.130772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:55.132339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 678/1342] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:16:55.138315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d1bc-9494-43a0-8897-dd6e4a251a24 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:55.138808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d1bc-9494-43a0-8897-dd6e4a251a24 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:55.139049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d1bc-9494-43a0-8897-dd6e4a251a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:55.209666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bc47d1bc-9494-43a0-8897-dd6e4a251a24 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:16:55.210980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bc47d1bc-9494-43a0-8897-dd6e4a251a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:55.311558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 665/1343] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:16:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:16:55.599828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:16:55.601398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:55.708999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593f21f-9576-4494-b2bf-e8305fb1bd13 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:16:55.781036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:16:55.781643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:16:55.781919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:16:55.782186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:16:55.783817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:16:55.828631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:16:55.895669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:16:55.897107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8cfc4f-98ed-423b-9831-10875a18da22 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:16:55.914866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 679/1344] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:16:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:00.000028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:00.000481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:00.000587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:00.001890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 666/1345] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:00.007540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a299b-2d83-42ed-b3d5-19bad7061baa None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:00.007960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a299b-2d83-42ed-b3d5-19bad7061baa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:00.008088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a299b-2d83-42ed-b3d5-19bad7061baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:00.073063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-412a299b-2d83-42ed-b3d5-19bad7061baa None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:00.074269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-412a299b-2d83-42ed-b3d5-19bad7061baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:00.168297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:00.170547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 680/1346] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:00.460687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:00.462345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:00.520059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:00.565904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5413f-a6c2-4867-b59f-02c864f1927a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:00.638712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:00.639303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:00.639582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:00.639850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:00.641448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:00.744179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:00.748032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:00.749465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7937f-73c3-46d2-ba3c-bdbbce9558e2 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:00.767554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 667/1347] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:05.724093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd091-0667-425d-a38d-3ad45d0b3416 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:05.724562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd091-0667-425d-a38d-3ad45d0b3416 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:05.724873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdd091-0667-425d-a38d-3ad45d0b3416 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:05.726870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 681/1348] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:05.732049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6631-0d53-4c9c-a50d-166194845463 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:05.732796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6631-0d53-4c9c-a50d-166194845463 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:05.733209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e6631-0d53-4c9c-a50d-166194845463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:05.820105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6e1e6631-0d53-4c9c-a50d-166194845463 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:05.822588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e1e6631-0d53-4c9c-a50d-166194845463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:06.020500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 668/1349] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:17:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:10.146993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e320620-8b00-443e-bbd6-1ed1df371fee None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:10.147531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e320620-8b00-443e-bbd6-1ed1df371fee None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:10.147531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e320620-8b00-443e-bbd6-1ed1df371fee None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:10.148875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 682/1350] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:10.154180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019aa2-c8fc-407b-9ccb-5d3184ddf8d6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:10.154638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019aa2-c8fc-407b-9ccb-5d3184ddf8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:10.154831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019aa2-c8fc-407b-9ccb-5d3184ddf8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:10.209433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:10.233219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a1019aa2-c8fc-407b-9ccb-5d3184ddf8d6 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:10.234438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a1019aa2-c8fc-407b-9ccb-5d3184ddf8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:10.349978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 669/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:17:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:11.827035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde068-28d2-48a1-bf41-e5c116f67dce None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:11.827561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde068-28d2-48a1-bf41-e5c116f67dce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:11.827658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcde068-28d2-48a1-bf41-e5c116f67dce None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:11.829170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 683/1352] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:11.834992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b2ae9-c25a-4bbf-9a19-8855fb1ef4c7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:11.835432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b2ae9-c25a-4bbf-9a19-8855fb1ef4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:11.836008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b2ae9-c25a-4bbf-9a19-8855fb1ef4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:11.913934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b93b2ae9-c25a-4bbf-9a19-8855fb1ef4c7 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:11.915170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b93b2ae9-c25a-4bbf-9a19-8855fb1ef4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:11.957224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:12.024456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 670/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:17:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:15.527431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5d4c5-bb7a-49db-bbc3-de265b563073 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:15.528352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5d4c5-bb7a-49db-bbc3-de265b563073 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:15.528832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5d4c5-bb7a-49db-bbc3-de265b563073 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:15.530937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 684/1354] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:15.536873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056fd3-3b2f-4884-9492-5e43585a86d4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:15.537319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056fd3-3b2f-4884-9492-5e43585a86d4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:15.537416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d056fd3-3b2f-4884-9492-5e43585a86d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:15.607907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9d056fd3-3b2f-4884-9492-5e43585a86d4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:15.609009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9d056fd3-3b2f-4884-9492-5e43585a86d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:15.709378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 671/1355] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:16.013797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:16.014085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:16.014535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:16.016095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 685/1356] 10.222.0.20 () {54 vars in 965 bytes} [Sun Sep 18 06:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:16.025645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a7a59-43e6-4688-86d3-8cd3f8e786c2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:16.026154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a7a59-43e6-4688-86d3-8cd3f8e786c2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:16.026874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a7a59-43e6-4688-86d3-8cd3f8e786c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:16.053306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.115676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ea1a7a59-43e6-4688-86d3-8cd3f8e786c2 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:16.116866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ea1a7a59-43e6-4688-86d3-8cd3f8e786c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:16.242174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 672/1357] 10.222.0.20 () {58 vars in 1062 bytes} [Sun Sep 18 06:17:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:16.250911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:16.252771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:16.405443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f86a6-943f-4bd1-9d1a-83022dd8cbe5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:16.484276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:16.526786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:16.527315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:16.527590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:16.528104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:16.529457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:16.623236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:16.624892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812824f8-31fc-4818-acc7-5b53856691d3 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:16.645110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 686/1358] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:17:19.045385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:19.045943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:19.045943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:19.047788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 673/1359] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:19.053685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba4315-ba8c-4d72-8d33-1b504414fa0b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:19.054156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba4315-ba8c-4d72-8d33-1b504414fa0b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:19.054286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba4315-ba8c-4d72-8d33-1b504414fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:19.122646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-11ba4315-ba8c-4d72-8d33-1b504414fa0b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:19.123963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-11ba4315-ba8c-4d72-8d33-1b504414fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:19.163948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:19.232414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 687/1360] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:19.532200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:19.533768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:19.604023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:19.704025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42ad5c-1b31-41e0-8f38-a9c13673ba12 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:19.848120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:19.848120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:19.848416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:19.848653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:19.850455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:19.874232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:19.945199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:19.946821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16260328-3949-49a5-beb8-23ec79237467 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:19.965481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 674/1361] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:22.458642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c255a8-6af3-4fb5-94d7-3acb5b342221 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:22.459727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c255a8-6af3-4fb5-94d7-3acb5b342221 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:22.460228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c255a8-6af3-4fb5-94d7-3acb5b342221 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:22.461820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 688/1362] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:22.468262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27143-1ed6-4365-9f15-f7bbd0fa16e9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:22.468718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27143-1ed6-4365-9f15-f7bbd0fa16e9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:22.468823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e27143-1ed6-4365-9f15-f7bbd0fa16e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:22.541187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f4e27143-1ed6-4365-9f15-f7bbd0fa16e9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:22.542389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f4e27143-1ed6-4365-9f15-f7bbd0fa16e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:22.641180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:22.648859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 675/1363] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:22.943685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:22.943993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:22.944408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:22.945857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 689/1364] 10.222.0.20 () {54 vars in 965 bytes} [Sun Sep 18 06:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:22.951777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78735b-2da5-40af-803b-3791f9c2e78a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:22.952134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78735b-2da5-40af-803b-3791f9c2e78a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:22.952464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78735b-2da5-40af-803b-3791f9c2e78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:23.033839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7e78735b-2da5-40af-803b-3791f9c2e78a None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:23.035067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7e78735b-2da5-40af-803b-3791f9c2e78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:23.163548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 676/1365] 10.222.0.20 () {58 vars in 1062 bytes} [Sun Sep 18 06:17:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:23.171132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:23.172638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:23.323058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d5632-b251-42c1-ab98-21bcd3b6cc38 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:23.343682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:23.454010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:23.455171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:23.455444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:23.455712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:23.457457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:23.558097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:23.559931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19719d0-dd01-42eb-972a-079f1df38c5c service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:23.580043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 690/1366] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:23.769505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b309-9cb5-4916-af26-50c5e666954a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:23.770164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b309-9cb5-4916-af26-50c5e666954a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:23.770164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b309-9cb5-4916-af26-50c5e666954a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:23.771696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 677/1367] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:23.777297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e5d9e-fbb6-4cf2-abf3-24e1f497002d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:23.777605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e5d9e-fbb6-4cf2-abf3-24e1f497002d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:23.777852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e5d9e-fbb6-4cf2-abf3-24e1f497002d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:23.817710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:23.862724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bb0e5d9e-fbb6-4cf2-abf3-24e1f497002d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:23.863654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb0e5d9e-fbb6-4cf2-abf3-24e1f497002d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:23.968605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 691/1368] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:24.301267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:24.301267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:24.301267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:24.301267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 678/1369] 10.222.0.20 () {54 vars in 964 bytes} [Sun Sep 18 06:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:24.306007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1aaa-95aa-41ed-850e-1f44c999d455 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:24.306289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1aaa-95aa-41ed-850e-1f44c999d455 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:24.306552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1aaa-95aa-41ed-850e-1f44c999d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:24.387820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-23aa1aaa-95aa-41ed-850e-1f44c999d455 None None] MFA Rules not processed for user `db771368757c449887be80383cceedbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:24.388299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-23aa1aaa-95aa-41ed-850e-1f44c999d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:24.436951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:24.478732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 692/1370] 10.222.0.20 () {58 vars in 1061 bytes} [Sun Sep 18 06:17:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:24.484601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:24.486772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:24.539866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:24.597934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6ecfb-7217-456f-a47a-78195e856ccb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:24.677061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:24.677794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:24.678168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:24.678543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:24.680328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:24.844068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:24.864167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:24.879439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617fb6a-410f-4176-bc32-1dc6885d8e23 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:24.937375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 679/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:25.849933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:25.850475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:25.850475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:25.851777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 693/1372] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:17:25.856950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf116bd-0db1-4926-8c8d-50ce19948f82 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:25.857381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf116bd-0db1-4926-8c8d-50ce19948f82 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:25.857638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf116bd-0db1-4926-8c8d-50ce19948f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:25.927250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-abf116bd-0db1-4926-8c8d-50ce19948f82 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:25.928471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-abf116bd-0db1-4926-8c8d-50ce19948f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:26.024366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 680/1373] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:26.319230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:26.320757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:26.482944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414817e-a312-4a98-a19b-f36e4d3b41d3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:26.539466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:26.604243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:26.604853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:26.605120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:26.605385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:26.607031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:26.699976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:26.701609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d43e6-c161-4a45-929c-424d7de5f9b3 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:26.721467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 694/1374] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:28.941537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:28.942102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:28.942102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:28.943586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 681/1375] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:28.949250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae4a7-ba32-4dbf-a506-8e6a8017f2ef None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:28.949656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae4a7-ba32-4dbf-a506-8e6a8017f2ef None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:28.949979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae4a7-ba32-4dbf-a506-8e6a8017f2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:29.019635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2a5ae4a7-ba32-4dbf-a506-8e6a8017f2ef None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:29.020831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2a5ae4a7-ba32-4dbf-a506-8e6a8017f2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:29.027912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:29.122620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 695/1376] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:29.416420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:29.417938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:29.494612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:29.583861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511d27-49ce-4ec2-b154-d597886a0672 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:29.700299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:29.701024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:29.701299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:29.701564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:29.703152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:29.723656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:29.803525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:29.805287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dcb50-03b9-4024-8520-51049dc8c7a8 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:29.826065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 682/1377] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:32.024297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:32.024884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:32.024884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:32.026469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 696/1378] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:32.032048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40c77-4d3b-43f7-a045-3ea18a2c92ce None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:32.032983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40c77-4d3b-43f7-a045-3ea18a2c92ce None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:32.033180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40c77-4d3b-43f7-a045-3ea18a2c92ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:32.102067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-49e40c77-4d3b-43f7-a045-3ea18a2c92ce None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:32.103299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-49e40c77-4d3b-43f7-a045-3ea18a2c92ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:32.202927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:32.210049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 683/1379] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:32.498498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:32.500074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:32.645515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:32.656787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a1ca-387b-473f-a858-98a56ef8a12d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:32.777689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:32.778181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:32.778477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:32.778793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:32.780529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:32.883416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:32.885038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c15a0-b373-4709-be13-54b2b77afe84 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:32.895158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:32.904383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 697/1380] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:35.408415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:35.408938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:35.408938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:35.410298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 684/1381] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:35.416539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c45ca-dc95-407f-9f7d-b4df090cb230 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:35.417064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c45ca-dc95-407f-9f7d-b4df090cb230 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:35.417064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c45ca-dc95-407f-9f7d-b4df090cb230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:35.485869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4f1c45ca-dc95-407f-9f7d-b4df090cb230 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:35.487122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4f1c45ca-dc95-407f-9f7d-b4df090cb230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:35.593179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 698/1382] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:35.958791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:35.960416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:36.118437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658590a7-6fb0-4ff2-bd2a-8423d6a40152 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:36.198523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:36.255669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:36.256522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:36.256871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:36.257225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:36.259371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:36.370246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:36.371887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987e287-31ba-4f3b-bb3f-8e2155337370 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:36.393211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 685/1383] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:38.930277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:38.930869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:38.930869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:38.935890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 699/1384] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:38.938463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1c22-5250-4012-92da-481be91b8eaf None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:38.938924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1c22-5250-4012-92da-481be91b8eaf None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:38.939027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae1c22-5250-4012-92da-481be91b8eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:39.013743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-06ae1c22-5250-4012-92da-481be91b8eaf None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:39.015103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-06ae1c22-5250-4012-92da-481be91b8eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:39.048403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:39.111635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 686/1385] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:39.407855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:39.409392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:39.499058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:39.568888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe7f74-1058-43d6-96e7-9f2c139f908e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:39.678257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:39.678895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:39.679176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:39.679442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:39.681039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:39.720664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:39.772213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:39.773835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe50593-5737-40a4-94ab-6f611b92c28a service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:39.801177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 700/1386] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:40.644196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd489d-f332-4d62-94cf-ebd831140455 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:40.644801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd489d-f332-4d62-94cf-ebd831140455 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:40.644801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd489d-f332-4d62-94cf-ebd831140455 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:40.646083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 687/1387] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:40.651304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4c58-84d6-4929-8384-3224a07fd6ee None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:40.651762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4c58-84d6-4929-8384-3224a07fd6ee None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:40.651860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e4c58-84d6-4929-8384-3224a07fd6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:40.720013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9f9e4c58-84d6-4929-8384-3224a07fd6ee None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:40.721193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9f9e4c58-84d6-4929-8384-3224a07fd6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:40.819231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 701/1388] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:41.118467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:41.118783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:41.119056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:41.120458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 688/1389] 10.222.0.20 () {54 vars in 962 bytes} [Sun Sep 18 06:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:41.126406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3a215-5d67-4feb-8cb5-30eeb0fc8b66 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:41.126726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3a215-5d67-4feb-8cb5-30eeb0fc8b66 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:41.126997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3a215-5d67-4feb-8cb5-30eeb0fc8b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:41.144590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:41.204487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-30b3a215-5d67-4feb-8cb5-30eeb0fc8b66 None None] MFA Rules not processed for user `2e5439bf8f1f480faef63316021d4b27`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:41.205704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-30b3a215-5d67-4feb-8cb5-30eeb0fc8b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:41.280451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 702/1390] 10.222.0.20 () {58 vars in 1059 bytes} [Sun Sep 18 06:17:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:41.288413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:41.289931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:41.394565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ca2f6-3789-4bbd-aa8c-6926d545370f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:41.454477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:41.474405 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:41.475047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:41.475322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:41.475592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:41.477189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:41.591860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:41.593316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d47ec-ed24-4897-8a80-0f0dd6081c95 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:41.612168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 689/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:42.029721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3aca-157a-4171-ba55-dbc1d5cc1876 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.030260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3aca-157a-4171-ba55-dbc1d5cc1876 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.030260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b3aca-157a-4171-ba55-dbc1d5cc1876 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.031791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 703/1392] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:42.038618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.039115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.039223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.113669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:42.114807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.157952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:42.220271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 690/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:42.373963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a101-4982-4500-8b9d-f10b31616cef None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.374264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a101-4982-4500-8b9d-f10b31616cef None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.374510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a101-4982-4500-8b9d-f10b31616cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.441684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-43d7a101-4982-4500-8b9d-f10b31616cef None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:42.442947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-43d7a101-4982-4500-8b9d-f10b31616cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.497938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:42.516214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:42.517454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.530903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 704/1394] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:42.578883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a689c-de17-47c2-aed3-72f3d885357a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.579268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a689c-de17-47c2-aed3-72f3d885357a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.579538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a689c-de17-47c2-aed3-72f3d885357a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.580921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 705/1395] 10.222.0.20 () {54 vars in 943 bytes} [Sun Sep 18 06:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:17:42.587001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.587278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.587844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.640824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34350e61-a5e8-4a0d-8e0e-d96450f8eb51 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:42.664628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] MFA Rules not processed for user `d763c798dc824e92b5e51deb0e059ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:42.665849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.697576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:42.757025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:42.757625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.757901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.758169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.759807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.773463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 706/1396] 10.222.0.20 () {58 vars in 1040 bytes} [Sun Sep 18 06:17:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:42.783146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:42.792593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92309a-0a8e-4e97-acf8-b761d0f973c7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:42.818507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:42.850449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:42.852109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e7fb-46df-45f6-9f44-22351c91e5cb service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:42.870779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 691/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:17:42.898439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:42.899227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:42.899495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:42.899762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:42.901408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:42.993779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:42.995480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24289001-2f07-45ab-8958-d66326b536a2 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:43.005510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 707/1398] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 18 06:17:45.078822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:45.079344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:45.079344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:45.080703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 692/1399] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:45.086477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb775-a7c6-409c-b695-f7793a7d721c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:45.087434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb775-a7c6-409c-b695-f7793a7d721c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:45.089580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fb775-a7c6-409c-b695-f7793a7d721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:45.162046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c32fb775-a7c6-409c-b695-f7793a7d721c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:45.163576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c32fb775-a7c6-409c-b695-f7793a7d721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:45.171272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:45.274688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 708/1400] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:45.566322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:45.567862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:45.639611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:45.737505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa4a54-7333-4d6d-bd20-c80478956240 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:45.866202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:45.866752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:45.867022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:45.867287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:45.868996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:45.903520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:45.976367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:45.977970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b90f43-f95f-415c-9822-7247d90fdb44 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:46.001629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 693/1401] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:48.284212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:48.284778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:48.284778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:48.290730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 709/1402] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:48.291960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86ef-f198-433c-b10f-1fb469f72350 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:48.292394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86ef-f198-433c-b10f-1fb469f72350 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:48.292498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f86ef-f198-433c-b10f-1fb469f72350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:48.362856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c92f86ef-f198-433c-b10f-1fb469f72350 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:48.364083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c92f86ef-f198-433c-b10f-1fb469f72350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:48.478619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 694/1403] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:17:48.772861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:48.774806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:48.923260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:48.934095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a3907-bfbf-4744-8604-9a3e912e8884 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:49.069474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:49.070095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:49.070371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:49.070670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:49.072315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:49.177462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:49.179581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71b2f2-5e9c-44a1-807b-43966e7d05e6 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:49.191289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:49.200785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 710/1404] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:17:51.403150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:51.403672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:51.403672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:51.405324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 695/1405] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:17:51.410604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865da6a6-dd24-4a85-af6c-ebc171a25aee None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:51.411010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865da6a6-dd24-4a85-af6c-ebc171a25aee None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:51.411202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865da6a6-dd24-4a85-af6c-ebc171a25aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:51.487057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-865da6a6-dd24-4a85-af6c-ebc171a25aee None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:17:51.488288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-865da6a6-dd24-4a85-af6c-ebc171a25aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:51.603098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 711/1406] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:17:51.891598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:17:51.893123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:51.895962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:52.067236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad42421-e76f-42bf-b20b-abfedbe24dc9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:17:52.151064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:17:52.194657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:17:52.195259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:17:52.195532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:17:52.195805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:17:52.197402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:17:52.319380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:17:52.321135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4866-0cfe-4cf7-bf75-3a9e8f6cce5d service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:17:52.352936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 696/1407] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:01.601554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=67 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:02.348714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=61 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:05.598802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec10053-f999-42c5-a3a6-a23807fd310c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:05.599356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec10053-f999-42c5-a3a6-a23807fd310c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:05.599356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec10053-f999-42c5-a3a6-a23807fd310c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:05.600742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 712/1408] 10.222.0.20 () {54 vars in 918 bytes} [Sun Sep 18 06:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:05.605857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5aea8-7a46-4804-bda5-04dd11d73a27 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:05.606290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5aea8-7a46-4804-bda5-04dd11d73a27 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:05.606414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5aea8-7a46-4804-bda5-04dd11d73a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:05.680970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-29d5aea8-7a46-4804-bda5-04dd11d73a27 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:05.682162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-29d5aea8-7a46-4804-bda5-04dd11d73a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:05.784221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 697/1409] 10.222.0.20 () {58 vars in 1015 bytes} [Sun Sep 18 06:18:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:05.798912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5365-7329-4dee-9c3c-2f58f8fec8c1 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:05.799188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5365-7329-4dee-9c3c-2f58f8fec8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:05.799451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f5365-7329-4dee-9c3c-2f58f8fec8c1 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:05.800798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 713/1410] 10.222.0.20 () {54 vars in 918 bytes} [Sun Sep 18 06:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:05.805435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ceba-9607-43d3-addd-f350ef7b7e0a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:05.805720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ceba-9607-43d3-addd-f350ef7b7e0a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:05.805992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99ceba-9607-43d3-addd-f350ef7b7e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:05.878904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:05.882972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dc99ceba-9607-43d3-addd-f350ef7b7e0a None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:05.884372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dc99ceba-9607-43d3-addd-f350ef7b7e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:06.000432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 698/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Sun Sep 18 06:18:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:08.549578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:08.550124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:08.550124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:08.551480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 714/1412] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:08.556667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ac4e-7ad2-4cae-986c-cabd01bcbd83 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:08.557092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ac4e-7ad2-4cae-986c-cabd01bcbd83 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:08.557218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ac4e-7ad2-4cae-986c-cabd01bcbd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:08.622353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-20b0ac4e-7ad2-4cae-986c-cabd01bcbd83 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:08.623596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-20b0ac4e-7ad2-4cae-986c-cabd01bcbd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:08.677735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:08.719905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 699/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:09.013435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:09.015167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:09.165116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6305b2f3-1134-4990-8c1a-8bd4d9bb5044 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:09.235466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:09.274657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:09.275220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:09.275497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:09.275758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:09.277371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:09.363182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:09.364784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fa3ad-cade-43fe-b37c-9bad744fc9da service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:09.383053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 715/1414] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:10.535335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:10.535867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:10.535867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:10.537269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 700/1415] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:10.543350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311bd9f-05a3-4f55-b98b-63998390ae79 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:10.543769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311bd9f-05a3-4f55-b98b-63998390ae79 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:10.543875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311bd9f-05a3-4f55-b98b-63998390ae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:10.617903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6311bd9f-05a3-4f55-b98b-63998390ae79 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:10.619107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6311bd9f-05a3-4f55-b98b-63998390ae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:10.645346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:10.717791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 716/1416] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:10.865956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:10.867524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:11.011334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab4fb1-901b-44c0-9c7e-26b63cf68928 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:11.051209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:11.122597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:11.123210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:11.123482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:11.123747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:11.125317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:11.213340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:11.214967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33235db3-da00-4647-8d66-13bb66f0fe5d service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:11.232451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 701/1417] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:12.350057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:12.350652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:12.350652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:12.351988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 717/1418] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:12.358253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae00c-d412-49b5-a131-a3bcec7320e3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:12.358703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae00c-d412-49b5-a131-a3bcec7320e3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:12.358844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ae00c-d412-49b5-a131-a3bcec7320e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:12.412873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:12.434920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-328ae00c-d412-49b5-a131-a3bcec7320e3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:12.436126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-328ae00c-d412-49b5-a131-a3bcec7320e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:12.533164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 702/1419] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:12.682872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:12.684471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:12.830405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4941f05-453a-4090-b763-a7f405f36635 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:12.842227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:12.940244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:12.940758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:12.941034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:12.941302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:12.942938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:13.030805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:13.032503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaebab7-71f5-4cd0-971e-8b0a2664716d service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:13.050546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 718/1420] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:14.145858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:14.146746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:14.146746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:14.147785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 703/1421] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:14.154206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1e983-2410-45cb-b6b4-d3abff72fe9b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:14.154680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1e983-2410-45cb-b6b4-d3abff72fe9b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:14.154784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1e983-2410-45cb-b6b4-d3abff72fe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:14.174117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:14.232748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-38b1e983-2410-45cb-b6b4-d3abff72fe9b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:14.233973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-38b1e983-2410-45cb-b6b4-d3abff72fe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:14.335010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 719/1422] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:14.484745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:14.486773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:14.596403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:14.633263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e753142-dbe3-4364-9f48-317a5318eef5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:14.743416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:14.743937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:14.744213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:14.744483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:14.746068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:14.824349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:14.838199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:14.839829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239d116-e66a-42f5-8fcc-485e3fad224e service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:14.858074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 704/1423] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:15.970233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:15.970822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:15.970822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:15.972181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 720/1424] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:15.978367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6ddca-c2cd-4151-bb36-f6e99c93ddc7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:15.978839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6ddca-c2cd-4151-bb36-f6e99c93ddc7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:15.978954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6ddca-c2cd-4151-bb36-f6e99c93ddc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:16.054089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ede6ddca-c2cd-4151-bb36-f6e99c93ddc7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:16.055321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ede6ddca-c2cd-4151-bb36-f6e99c93ddc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:16.152149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 705/1425] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:16.301945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:16.303630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:16.387758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:16.448620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861de8bb-8b89-4190-a01e-88d5f4d38d52 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:16.558210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:16.558838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:16.559112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:16.559376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:16.560967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:16.608835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:16.648523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:16.650152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb7fa1-8f93-4ebc-ac22-03ff2a83597a service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:16.668111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 721/1426] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:17.722120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:17.722703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:17.722703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:17.723947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 706/1427] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:17.731485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3964ba-b07d-41f1-80a8-814ddf54a000 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:17.731921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3964ba-b07d-41f1-80a8-814ddf54a000 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:17.732024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3964ba-b07d-41f1-80a8-814ddf54a000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:17.809813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5f3964ba-b07d-41f1-80a8-814ddf54a000 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:17.811050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f3964ba-b07d-41f1-80a8-814ddf54a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:18.020144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 722/1428] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:18.168634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:18.170168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:18.220487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:18.316038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6accb3f-0fc0-4a2e-9f7d-e3ca462efce5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:18.426003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:18.426619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:18.426892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:18.427157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:18.428746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:18.435438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:18.519529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:18.521136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c77109-ab3e-4cb3-a185-b9c76e5cdeb5 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:18.539328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 707/1429] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:19.582397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:19.582939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:19.582939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:19.584245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 723/1430] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:19.590529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843228f-fff5-49ab-ab1a-a3a957a715dd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:19.590939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843228f-fff5-49ab-ab1a-a3a957a715dd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:19.591097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843228f-fff5-49ab-ab1a-a3a957a715dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:19.667524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4843228f-fff5-49ab-ab1a-a3a957a715dd None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:19.668721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4843228f-fff5-49ab-ab1a-a3a957a715dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:19.735354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:19.764138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 708/1431] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:19.913478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:19.915038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:19.932456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:20.055549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245114be-480e-4364-bc7f-140d0a7b04d1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:20.138287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:20.164388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:20.164925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:20.165196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:20.165462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:20.167093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:20.253016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:20.254623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1651c8-6ebe-4e34-a034-752437dcb2ac service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:20.272043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 724/1432] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:21.306924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:21.307452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:21.307452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:21.308748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 709/1433] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:21.315324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72bdb-acd4-491f-878b-d350e3211f69 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:21.315719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72bdb-acd4-491f-878b-d350e3211f69 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:21.315855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72bdb-acd4-491f-878b-d350e3211f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:21.390259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-edc72bdb-acd4-491f-878b-d350e3211f69 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:21.391576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-edc72bdb-acd4-491f-878b-d350e3211f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:21.431397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:21.484339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 725/1434] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:21.634943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:21.636454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:21.782380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc6db6-9e25-47df-8c13-3a29b90d8946 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:21.834145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:21.891950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:21.892491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:21.892747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:21.893007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:21.894623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:21.981408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:21.983026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c9dfb-0b67-47b6-9963-caa2c5e5b151 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:22.000928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 710/1435] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:23.043541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:23.043970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:23.044092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:23.045464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 726/1436] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:23.051701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562fe6c-5db1-46ff-9d60-018b62905ac0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:23.052144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562fe6c-5db1-46ff-9d60-018b62905ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:23.052248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562fe6c-5db1-46ff-9d60-018b62905ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:23.125355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0562fe6c-5db1-46ff-9d60-018b62905ac0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:23.127060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0562fe6c-5db1-46ff-9d60-018b62905ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:23.129093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:23.223125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 711/1437] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:23.370614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:23.372065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:23.517061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf92c53-20bb-496d-979b-fed1456c8af7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:23.543042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:23.626176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:23.626738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:23.627028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:23.627272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:23.628853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:23.714471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:23.716093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118c07a-3d2e-4fe9-8056-5e69943695f5 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:23.733410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 727/1438] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:25.774816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d642-abec-4329-9315-c337c833952f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:25.775354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d642-abec-4329-9315-c337c833952f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:25.775354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d642-abec-4329-9315-c337c833952f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:25.776692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 712/1439] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:25.782727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f91b80-e413-4f17-94a8-1a1e9cf06098 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:25.783173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f91b80-e413-4f17-94a8-1a1e9cf06098 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:25.783332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f91b80-e413-4f17-94a8-1a1e9cf06098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:25.824268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:25.848896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-28f91b80-e413-4f17-94a8-1a1e9cf06098 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:25.850099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-28f91b80-e413-4f17-94a8-1a1e9cf06098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:25.947009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 728/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:26.232974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69d642-abec-4329-9315-c337c833952f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:26.234555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ea69d642-abec-4329-9315-c337c833952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:26.337979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69d642-abec-4329-9315-c337c833952f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:26.369335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:26.411026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:26.411621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:26.411896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:26.412167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:26.413758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:26.521369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:26.522783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595e7c9-d1c4-433d-ad04-4331846aec8b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:26.540584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 713/1441] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:28.804441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:28.804983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:28.804983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:28.806261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 729/1442] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:28.811411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb14d1-9f5c-437e-bebc-2a44ed1a3423 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:28.811858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb14d1-9f5c-437e-bebc-2a44ed1a3423 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:28.811965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb14d1-9f5c-437e-bebc-2a44ed1a3423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:28.877192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-62cb14d1-9f5c-437e-bebc-2a44ed1a3423 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:28.878411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-62cb14d1-9f5c-437e-bebc-2a44ed1a3423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:28.891775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:28.976254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 714/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:29.255389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:29.256925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:29.358030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30858dc-ccb1-4853-8060-3db46f532e98 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:29.371540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:29.430969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:29.431568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:29.431839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:29.432100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:29.433679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:29.540006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:29.541445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ae5c8-0e95-43a7-98da-a1c4ca2f5c32 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:29.559416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 730/1444] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:31.991051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:31.991621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:31.991621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:31.992922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 715/1445] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:31.998130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c578-6ccc-4705-b3b5-4280f966344b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:31.998625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c578-6ccc-4705-b3b5-4280f966344b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:31.998736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6c578-6ccc-4705-b3b5-4280f966344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:32.046465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:32.065022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a0c6c578-6ccc-4705-b3b5-4280f966344b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:32.066206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a0c6c578-6ccc-4705-b3b5-4280f966344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:32.159750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 731/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:32.444490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:32.446012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:32.551171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf854b-3afb-499e-87f9-f20f877503b0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:32.591826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:32.626990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:32.627571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:32.627866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:32.628116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:32.629717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:32.739809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:32.741156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2280d-2c74-4b64-831c-daf7541e9ec5 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:32.759547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 716/1447] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:35.051203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:35.051637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:35.051763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:35.053114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 732/1448] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:35.058346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4c07-2ad5-4805-a037-d79b99408adc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:35.058794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4c07-2ad5-4805-a037-d79b99408adc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:35.058908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf4c07-2ad5-4805-a037-d79b99408adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:35.123379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-abbf4c07-2ad5-4805-a037-d79b99408adc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:35.124569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-abbf4c07-2ad5-4805-a037-d79b99408adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:35.146917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:35.224619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 717/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:35.510920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:35.512494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:35.612211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fccbf-6983-4bff-ad82-8f80d565c129 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:35.631665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:35.683839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:35.684432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:35.684704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:35.684972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:35.686544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:35.790861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:35.792220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621619c-2a6b-44ef-a1ff-391909bc8cad service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:35.809418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 733/1450] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:38.116281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:38.116831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:38.116831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:38.118153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 718/1451] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:38.123699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d23e1-eb8b-49e2-aff2-ad4f301a808f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:38.124130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d23e1-eb8b-49e2-aff2-ad4f301a808f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:38.124517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d23e1-eb8b-49e2-aff2-ad4f301a808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:38.204316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:38.208359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6f2d23e1-eb8b-49e2-aff2-ad4f301a808f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:38.209592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f2d23e1-eb8b-49e2-aff2-ad4f301a808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:38.305265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 734/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:38.586095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:38.587686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:38.691717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d519-50f5-4f8f-a496-2ce8469a7f8d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:38.741454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:38.766942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:38.767541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:38.767820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:38.768087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:38.769673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:38.876201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:38.877625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffefdb-e134-487f-a504-c33d3a737af1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:38.895890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 719/1453] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:41.093797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:41.094993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:41.094993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:41.096434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 735/1454] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:41.103115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba510fd-1ca9-4333-8279-6124c9dc5dec None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:41.103561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba510fd-1ca9-4333-8279-6124c9dc5dec None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:41.103685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba510fd-1ca9-4333-8279-6124c9dc5dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:41.169039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1ba510fd-1ca9-4333-8279-6124c9dc5dec None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:41.170240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1ba510fd-1ca9-4333-8279-6124c9dc5dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:41.198785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:41.266796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 720/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:41.548357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:41.549933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:41.649924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01ab13-e704-4ed1-bd9b-fbd21c407ba3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:41.677658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:41.722248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:41.722812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:41.723085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:41.723351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:41.724953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:41.831007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:41.832438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aae794-6813-43f9-95d1-3e8a9f14f918 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:41.850439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 736/1456] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:44.050299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:44.051001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:44.051001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:44.052205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 721/1457] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:44.057694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f407b42-b146-48f9-8e9d-4fbeee5b7df7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:44.058100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f407b42-b146-48f9-8e9d-4fbeee5b7df7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:44.058249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f407b42-b146-48f9-8e9d-4fbeee5b7df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:44.122483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6f407b42-b146-48f9-8e9d-4fbeee5b7df7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:44.123725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f407b42-b146-48f9-8e9d-4fbeee5b7df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:44.132272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:44.219307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 737/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:44.511661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:44.513299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:44.615795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55446c-3ca4-42c7-839c-edff3d20a255 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:44.670073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:44.688960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:44.689566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:44.689844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:44.690116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:44.691750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:44.918296 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:44.919765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957cc74-2428-4415-9363-d7b4719bdca3 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:44.938030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 722/1459] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:18:47.126875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:47.127417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:47.127417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:47.128751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 738/1460] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:47.134205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba8b26-15a8-47e1-8706-27780e8eae4c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:47.134667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba8b26-15a8-47e1-8706-27780e8eae4c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:47.134779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba8b26-15a8-47e1-8706-27780e8eae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:47.198441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-84ba8b26-15a8-47e1-8706-27780e8eae4c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:47.199671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-84ba8b26-15a8-47e1-8706-27780e8eae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:47.236808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:47.293979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 723/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:47.575083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:47.576632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:47.676812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18874a7b-cbe8-474f-973b-a5fbf982beb4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:47.713866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:47.748457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:47.749064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:47.749343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:47.749615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:47.751809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:47.859536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:47.861539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd179131-0b50-4fe8-b0ec-d55e4d70ca8d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:47.881426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 739/1462] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:50.038826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:50.039394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:50.039394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:50.040713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 724/1463] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:50.046166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625f2-8b06-4f16-8d81-3edaf2690331 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:50.046559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625f2-8b06-4f16-8d81-3edaf2690331 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:50.046745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c625f2-8b06-4f16-8d81-3edaf2690331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:50.112058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-03c625f2-8b06-4f16-8d81-3edaf2690331 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:50.113301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-03c625f2-8b06-4f16-8d81-3edaf2690331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:50.130556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:50.210482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 740/1464] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:50.493985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:50.495559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:50.598126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ec74f-44aa-4df1-b68f-37ee10b93887 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:50.664938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:50.672110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:50.672723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:50.672999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:50.673269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:50.674971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:50.785431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:50.786809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76b43a-b0c2-45df-9d0f-8e70523c7760 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:50.804173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 725/1465] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:18:52.940562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:52.941103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:52.941103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:52.942459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 741/1466] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:18:52.947724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56257306-f566-499f-94e8-2ad705ab124e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:52.948157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56257306-f566-499f-94e8-2ad705ab124e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:52.948270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56257306-f566-499f-94e8-2ad705ab124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:53.013026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56257306-f566-499f-94e8-2ad705ab124e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:18:53.014214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56257306-f566-499f-94e8-2ad705ab124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:53.059970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:53.107321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 726/1467] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:18:53.390658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:18:53.392236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:53.493415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6de4ea-154c-4ee2-9faa-3a99fe8f6920 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:18:53.542821 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:18:53.567746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:18:53.568375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:18:53.568644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:18:53.568907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:18:53.570492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:18:53.676080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:18:53.677505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8557f-6472-4eb3-8891-79e4a9c2b0ce service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:18:53.694881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 742/1468] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:03.105799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=22 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:03.692637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=66 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:13.536879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9a8d-fb74-4a3b-8ec4-1b6d263b7c0b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:13.537469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9a8d-fb74-4a3b-8ec4-1b6d263b7c0b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:13.537469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb9a8d-fb74-4a3b-8ec4-1b6d263b7c0b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:13.538809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 727/1469] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:13.543974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2193-982b-40b9-86dd-285bea61fd81 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:13.544381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2193-982b-40b9-86dd-285bea61fd81 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:13.544516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b2193-982b-40b9-86dd-285bea61fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:13.607910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b04b2193-982b-40b9-86dd-285bea61fd81 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:13.609151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b04b2193-982b-40b9-86dd-285bea61fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:13.703040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 743/1470] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:14.002181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:14.002467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:14.002772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:14.004097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 728/1471] 10.222.0.20 () {54 vars in 962 bytes} [Sun Sep 18 06:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:14.009323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceedfe-a8ca-4f8f-8265-c0d44017124d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:14.009632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceedfe-a8ca-4f8f-8265-c0d44017124d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:14.009905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ceedfe-a8ca-4f8f-8265-c0d44017124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:14.083640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-77ceedfe-a8ca-4f8f-8265-c0d44017124d None None] MFA Rules not processed for user `2e5439bf8f1f480faef63316021d4b27`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:14.084836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-77ceedfe-a8ca-4f8f-8265-c0d44017124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:14.104671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:14.159069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 744/1472] 10.222.0.20 () {58 vars in 1059 bytes} [Sun Sep 18 06:19:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:14.166917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:14.168465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:14.270308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f33a0-6b77-4d1e-9e9a-2e11727d5bfc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:14.345820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:14.346421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:14.346732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:14.347003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:14.348600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:14.403718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:14.458743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:14.460209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4eb2e-10a4-4315-83f1-0f438ced7556 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:14.480016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 729/1473] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:16.705713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:16.706222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:16.706344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:16.707727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 745/1474] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:16.712952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0349ac7-fee4-4d07-89e4-12b661965e2e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:16.713337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0349ac7-fee4-4d07-89e4-12b661965e2e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:16.713495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0349ac7-fee4-4d07-89e4-12b661965e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:16.778112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a0349ac7-fee4-4d07-89e4-12b661965e2e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:16.779291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a0349ac7-fee4-4d07-89e4-12b661965e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:16.873761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 730/1475] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:17.151428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:17.152960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:17.288823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146fb33-5786-4a1f-bbd5-47a38f5bd426 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:17.336402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:17.379634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:17.380242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:17.380521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:17.380792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:17.381431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:17.383173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:19:17.383173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:19:17.383173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:19:17.384581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bf15a-f003-4ae0-b1bd-9ffa59c77f61 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:17.403334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 746/1476] 10.222.0.20 () {56 vars in 1108 bytes} [Sun Sep 18 06:19:17 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:19:19.569485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:19.570037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:19.570037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:19.571368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 731/1477] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:19.576844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f306ab-77f1-4423-aa64-0be7b7244c12 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:19.577310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f306ab-77f1-4423-aa64-0be7b7244c12 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:19.577416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f306ab-77f1-4423-aa64-0be7b7244c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:19.642594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b9f306ab-77f1-4423-aa64-0be7b7244c12 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:19.643830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b9f306ab-77f1-4423-aa64-0be7b7244c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:19.741555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 747/1478] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:20.040280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:20.041819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:20.051576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:20.185999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d220129-75f8-44eb-ab3f-9095176e270a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:20.270239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:20.301995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:20.302530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:20.302827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:20.303090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:20.304680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:20.394418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:20.396046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071765b0-e027-410e-884b-5a71b2124075 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:20.413626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 732/1479] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:22.754368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:22.754922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:22.754922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:22.756202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 748/1480] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:22.761791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecdf97-49ca-4a18-8519-c185a7e669e6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:22.762215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecdf97-49ca-4a18-8519-c185a7e669e6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:22.762344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecdf97-49ca-4a18-8519-c185a7e669e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:22.826353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2eecdf97-49ca-4a18-8519-c185a7e669e6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:22.827579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2eecdf97-49ca-4a18-8519-c185a7e669e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:22.874864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:22.924344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 733/1481] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:19:23.209541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:23.211132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:23.234538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:23.312024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2071-932e-4566-b4c9-44fae8d143af None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:23.382664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:23.383265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:23.383540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:23.384072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:23.385663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:23.458904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:23.493795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:23.495271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88753af-11c9-460b-9191-e8ab33abdde3 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:23.513002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 749/1482] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:25.749065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:25.749604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:25.749604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:25.750910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 734/1483] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:25.756182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44391b-df19-4aeb-adaf-f8190971fceb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:25.756595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44391b-df19-4aeb-adaf-f8190971fceb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:25.756734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44391b-df19-4aeb-adaf-f8190971fceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:25.821442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bc44391b-df19-4aeb-adaf-f8190971fceb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:25.822820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bc44391b-df19-4aeb-adaf-f8190971fceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:25.928113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 750/1484] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:26.224521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:26.226054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:26.326166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50294437-3b5b-4e1a-805e-a704c7df2c7a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:26.398431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:26.400362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:26.400972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:26.401437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:26.401866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:26.403722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:26.510820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:26.512390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da40643-6697-43ff-b08e-40aef313b8eb service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:26.534532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 735/1485] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:28.764743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:28.765273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:28.765273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:28.766606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 751/1486] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:28.771992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a18d8-36cc-4a97-8931-ecf7e3659bf0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:28.772391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a18d8-36cc-4a97-8931-ecf7e3659bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:28.772541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a18d8-36cc-4a97-8931-ecf7e3659bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:28.837402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-504a18d8-36cc-4a97-8931-ecf7e3659bf0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:28.838632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-504a18d8-36cc-4a97-8931-ecf7e3659bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:28.891360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:28.935608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 736/1487] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:29.229008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:29.230569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:29.267041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:29.332409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bce11-ed1f-4cc0-9c3e-212e5370093e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:29.405172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:29.405777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:29.406056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:29.406331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:29.408515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:29.487452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:29.517170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:29.518641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0d287-dc85-4bc9-ac37-95a1784ca573 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:29.536634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 752/1488] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:31.880540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:31.881070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:31.881070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:31.882343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 737/1489] 10.222.0.20 () {54 vars in 927 bytes} [Sun Sep 18 06:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:19:31.888016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01449c-acc6-4a25-834a-ee4a9d8747ee None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:31.888468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01449c-acc6-4a25-834a-ee4a9d8747ee None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:31.888566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01449c-acc6-4a25-834a-ee4a9d8747ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:31.952654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6f01449c-acc6-4a25-834a-ee4a9d8747ee None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:19:31.953853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f01449c-acc6-4a25-834a-ee4a9d8747ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:32.046409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 753/1490] 10.222.0.20 () {58 vars in 1024 bytes} [Sun Sep 18 06:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:19:32.340681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:19:32.342229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:32.443938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68879c-3bb9-4b2b-b6bf-34e7d9cbbecc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:19:32.517625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:19:32.518218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:19:32.518492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:19:32.518800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:19:32.520391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:19:32.528706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:32.630333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:19:32.631694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56318853-8899-4459-af93-bb0514c515b9 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:19:32.650546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 738/1491] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:19:42.044915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=82 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:19:42.648315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=50 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:52.271589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:52.272190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:52.272190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:52.347879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:20:52.349085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:52.445596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 754/1492] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:20:52.453726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:52.454227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:52.454340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:52.528192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:20:52.529325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:52.622748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 739/1493] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:20:52.627515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:52.628728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:52.640152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b4bc-6352-4343-b496-3bc6c272628a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:52.699468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:52.732406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:52.733004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:52.733274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:52.733535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:52.734171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:20:52.734466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:52.735497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:52.735497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:20:52.735497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:20:52.736376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:52.747006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 755/1494] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:20:52.755487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:52.765256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb41f5-e936-4537-8e91-0aae3d024c25 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:52.824956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:52.858649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:52.859245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:52.859517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:52.859780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:52.860398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:52.862108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:52.862108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:20:52.862108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:20:52.863317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:52.917555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 740/1495] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:20:52 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:20:52.921709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:52.923372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:52.956021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5a508-f3f3-4334-a86b-ebd03b1f8f03 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:53.050105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:53.050757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.051035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.051302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.051986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:53.053017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:53.053017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:20:53.053017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:20:53.054223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:53.096470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 756/1496] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:20:52 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:20:53.100446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:53.101964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.166385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca52b-7d68-4b6e-a079-be559143e5ce admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:53.247800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:53.260220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:53.260812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.261083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.261345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.261981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:53.263031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:53.263031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:20:53.263031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:20:53.263906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:53.276464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 741/1497] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:20:53.280494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:53.282044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.293439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c897-ab5b-4173-89e5-994fe8c22e8c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:53.318432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:53.387876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:53.388405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.388681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.388947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.389613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:53.391227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:53.391227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:20:53.391227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:20:53.392110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1dd62-6b49-4e52-90f3-0ae5f28d35fc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:53.404208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 757/1498] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:20:53.408355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:53.409922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.442649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e5320-c2db-4ca8-9851-d2f722ab0c5d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:53.535718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:53.536353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.536625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.536894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] PATH_INFO: `/v3/projects/6ea342b78df645f9aa69c7a5181ca8ec/users/1343c11cee4e48d4ae18638ca6d12164/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.558982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea342b78df645f9aa69c7a5181ca8ec, user_id=1343c11cee4e48d4ae18638ca6d12164, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:53.559991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:53.559991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:20:53.559991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:20:53.562586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b8579-8014-404c-972f-c85ecbc18a59 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:53.585899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 742/1499] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:20:53 2022] PUT /identity/v3/projects/6ea342b78df645f9aa69c7a5181ca8ec/users/1343c11cee4e48d4ae18638ca6d12164/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:20:53.597733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32ae9-71d3-4f80-a1ba-ff3184046d3c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.598109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32ae9-71d3-4f80-a1ba-ff3184046d3c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.598409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e32ae9-71d3-4f80-a1ba-ff3184046d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.654421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a9e32ae9-71d3-4f80-a1ba-ff3184046d3c None None] MFA Rules not processed for user `1343c11cee4e48d4ae18638ca6d12164`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:20:53.655708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a9e32ae9-71d3-4f80-a1ba-ff3184046d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.731379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 758/1500] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:20:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:20:53.744964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.745246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.745520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.746889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 743/1501] 10.222.0.20 () {54 vars in 964 bytes} [Sun Sep 18 06:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:20:53.752294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ec26-27f5-4ac1-b094-ba89d04e013f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:53.752585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ec26-27f5-4ac1-b094-ba89d04e013f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:53.752861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ec26-27f5-4ac1-b094-ba89d04e013f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:53.783583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:53.827375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fcd5ec26-27f5-4ac1-b094-ba89d04e013f None None] MFA Rules not processed for user `db771368757c449887be80383cceedbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:20:53.828576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fcd5ec26-27f5-4ac1-b094-ba89d04e013f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.901336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 759/1502] 10.222.0.20 () {58 vars in 1061 bytes} [Sun Sep 18 06:20:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:20:53.909591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:53.911102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:53.959946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:54.014667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86015f90-6d9d-4c2d-8ef6-1467262918fd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:54.087236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:54.087829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:54.088101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:54.088365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:54.089944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:54.186834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:54.188183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:54.191614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:54.206791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 744/1503] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:20:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:20:54.251265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683baf5-646e-49bd-8ccc-169a241f2af6 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:54.251551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683baf5-646e-49bd-8ccc-169a241f2af6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:54.251820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683baf5-646e-49bd-8ccc-169a241f2af6 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:54.253076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 760/1504] 10.222.0.20 () {52 vars in 830 bytes} [Sun Sep 18 06:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:20:54.256593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:54.257804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:54.354912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7d58-26a1-44dd-8c7c-2adf3420b4de service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:54.446288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:54.446874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:54.447152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:54.447422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] PATH_INFO: `/v3/users/1343c11cee4e48d4ae18638ca6d12164` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:54.460395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1343c11cee4e48d4ae18638ca6d12164)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:54.461413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:54.461413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:20:54.461413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:20:54.462721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:54.473030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:54.528623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1343c11cee4e48d4ae18638ca6d12164'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:54.537584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1343c11cee4e48d4ae18638ca6d12164'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:54.538143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1343c11cee4e48d4ae18638ca6d12164'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:54.538522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1343c11cee4e48d4ae18638ca6d12164'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:54.552833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1343c11cee4e48d4ae18638ca6d12164'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:54.580597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 745/1505] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:20:54 2022] DELETE /identity/v3/users/1343c11cee4e48d4ae18638ca6d12164 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:20:54.584913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:54.585389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:54.585494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:54.641484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:20:54.642705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:54.717143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:54.730307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 761/1506] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:20:54.740458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:54.741724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:54.840192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77769018-96e4-42a7-983e-f53263aa354b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:54.895345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:20:54.916233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:54.916834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:54.917111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:54.917382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:54.919023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:55.028085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:55.029746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce73ae0-2e99-4c27-9ee9-c929c1e3ed91 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:55.048242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 746/1507] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:20:55.769123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:20:55.770368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:20:55.781823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef107a-8532-437d-a21a-5bf5b5a679bb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:20:55.880914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:20:55.881548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:20:55.881846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:20:55.882111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] PATH_INFO: `/v3/projects/6ea342b78df645f9aa69c7a5181ca8ec` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:20:55.892037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea342b78df645f9aa69c7a5181ca8ec)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:20:55.893058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:20:55.893058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:20:55.893058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:20:55.894249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:20:55.952713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea342b78df645f9aa69c7a5181ca8ec', 'request_id': 'req-f0e75973-a440-4a5d-9330-0530ed54319d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:55.965983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea342b78df645f9aa69c7a5181ca8ec', 'request_id': 'req-f0e75973-a440-4a5d-9330-0530ed54319d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:55.974928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea342b78df645f9aa69c7a5181ca8ec', 'request_id': 'req-f0e75973-a440-4a5d-9330-0530ed54319d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:55.997301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] The token cache is being invalidate because project 6ea342b78df645f9aa69c7a5181ca8ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:20:55.997590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:55.998096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f0e75973-a440-4a5d-9330-0530ed54319d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:20:55.999974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 762/1508] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:20:55 2022] DELETE /identity/v3/projects/6ea342b78df645f9aa69c7a5181ca8ec => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:21:05.046412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=62 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:21:05.995351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:19.663125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:19.663734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:19.663734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:19.665109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 747/1509] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:19.670625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414ad0-74ff-432e-b6c3-391f392820fc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:19.671045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414ad0-74ff-432e-b6c3-391f392820fc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:19.671169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99414ad0-74ff-432e-b6c3-391f392820fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:19.751140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-99414ad0-74ff-432e-b6c3-391f392820fc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:19.752285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-99414ad0-74ff-432e-b6c3-391f392820fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:19.853716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 763/1510] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:20.008719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:20.011132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:20.134357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f634b-a133-4c76-bf01-f81f330e209a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:20.140036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:20.140658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:20.140931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:20.141197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:20.149038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:20.150640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb39f91-247a-4e9f-b57a-2250037c48be service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:20.160507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 748/1511] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:21.228903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbde2d-acf0-460c-8bfe-ec7ec9eee0b8 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:21.229436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbde2d-acf0-460c-8bfe-ec7ec9eee0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:21.229534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbde2d-acf0-460c-8bfe-ec7ec9eee0b8 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:21.230881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 764/1512] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:21.237440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fd728-0cb9-4a5d-93fc-5c1d0aef796a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:21.237901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fd728-0cb9-4a5d-93fc-5c1d0aef796a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:21.238005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fd728-0cb9-4a5d-93fc-5c1d0aef796a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:21.257151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-428fd728-0cb9-4a5d-93fc-5c1d0aef796a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:21.258352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-428fd728-0cb9-4a5d-93fc-5c1d0aef796a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:21.266996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 749/1513] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:21.424757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbde2d-acf0-460c-8bfe-ec7ec9eee0b8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:21.441639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbde2d-acf0-460c-8bfe-ec7ec9eee0b8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:21.447695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:21.448332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:21.448723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:21.449212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:21.458399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:21.460041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947f107-7643-4447-b673-180ac9f3cd20 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:21.470041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 765/1514] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:21.837654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27669-30d7-4eec-bb3b-58ac37c0f5db None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:21.837994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27669-30d7-4eec-bb3b-58ac37c0f5db None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:21.838193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27669-30d7-4eec-bb3b-58ac37c0f5db None None] PATH_INFO: `/v3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:21.839533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 750/1515] 10.222.0.27 () {54 vars in 925 bytes} [Sun Sep 18 06:22:21 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:22:21.846728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aba28-3906-4a54-a1e1-a7ae2f6fe832 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:21.847016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aba28-3906-4a54-a1e1-a7ae2f6fe832 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:21.847291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aba28-3906-4a54-a1e1-a7ae2f6fe832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:21.905642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-518aba28-3906-4a54-a1e1-a7ae2f6fe832 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:21.906884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-518aba28-3906-4a54-a1e1-a7ae2f6fe832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:21.923717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:21.977275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 766/1516] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 18 06:22:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:25.516840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1bc20-a0ef-484b-b902-17358417ca59 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:25.517331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1bc20-a0ef-484b-b902-17358417ca59 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:25.517429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1bc20-a0ef-484b-b902-17358417ca59 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:25.518766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 751/1517] 10.222.0.20 () {54 vars in 961 bytes} [Sun Sep 18 06:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:25.526474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7944-f574-4d70-81ba-14a8c9ba10b9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:25.526974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7944-f574-4d70-81ba-14a8c9ba10b9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:25.527090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7944-f574-4d70-81ba-14a8c9ba10b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:25.547155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-59ec7944-f574-4d70-81ba-14a8c9ba10b9 None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:25.548481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-59ec7944-f574-4d70-81ba-14a8c9ba10b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:25.557089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 767/1518] 10.222.0.20 () {58 vars in 1058 bytes} [Sun Sep 18 06:22:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:22:25.565391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1bc20-a0ef-484b-b902-17358417ca59 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:25.579359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1bc20-a0ef-484b-b902-17358417ca59 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:25.584366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:25.584949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:25.585227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:25.585497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:25.593686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:25.595299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d03a45-b2d4-4f54-8458-a9852eea16ff service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:25.605317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 752/1519] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:32.937457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07afd1-7877-40a8-882a-386b1eb6fe11 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:32.938064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07afd1-7877-40a8-882a-386b1eb6fe11 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:32.938064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07afd1-7877-40a8-882a-386b1eb6fe11 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:32.939419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 768/1520] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:32.945536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1a23d-3375-4d90-b77d-2261c49bc034 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:32.946009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1a23d-3375-4d90-b77d-2261c49bc034 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:32.946115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1a23d-3375-4d90-b77d-2261c49bc034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:32.966085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0fe1a23d-3375-4d90-b77d-2261c49bc034 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:32.967298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0fe1a23d-3375-4d90-b77d-2261c49bc034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:32.974934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 753/1521] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:33.127779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07afd1-7877-40a8-882a-386b1eb6fe11 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:33.143066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07afd1-7877-40a8-882a-386b1eb6fe11 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:33.148362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:33.148944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:33.149219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:33.149487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:33.157657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:33.159567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32212e-4bb1-4b19-acd8-11ab7fa19457 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:33.169832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 769/1522] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:34.229371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52122-db92-4cb6-a1d4-2fe6ca538126 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:34.229926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52122-db92-4cb6-a1d4-2fe6ca538126 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:34.230042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a52122-db92-4cb6-a1d4-2fe6ca538126 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:34.231346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 754/1523] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 18 06:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:34.237352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e95-e95a-4f47-bf14-5dc3bfc9bdea None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:34.237741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e95-e95a-4f47-bf14-5dc3bfc9bdea None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:34.237889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d06e95-e95a-4f47-bf14-5dc3bfc9bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:34.256631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-62d06e95-e95a-4f47-bf14-5dc3bfc9bdea None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:34.257818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-62d06e95-e95a-4f47-bf14-5dc3bfc9bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:34.266515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 770/1524] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 18 06:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:34.416347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a52122-db92-4cb6-a1d4-2fe6ca538126 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:34.430546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a52122-db92-4cb6-a1d4-2fe6ca538126 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:34.436481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:34.437057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:34.437331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:34.437596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:34.445547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:34.447154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b933e2-3bd5-4137-ae48-d004acbda7f0 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:34.457200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 755/1525] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:38.341436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:38.342013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:38.342013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:38.343342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 771/1526] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:38.348557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903e4b-dfd6-41f0-80be-a3895a5267d6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:38.348940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903e4b-dfd6-41f0-80be-a3895a5267d6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:38.349108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb903e4b-dfd6-41f0-80be-a3895a5267d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:38.368317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bb903e4b-dfd6-41f0-80be-a3895a5267d6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:38.369513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bb903e4b-dfd6-41f0-80be-a3895a5267d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:38.377846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 756/1527] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:38.664847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:38.666618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:38.743517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af479cf2-458c-4b9c-8db1-448890fcb439 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:38.747682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:38.748275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:38.748547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:38.748813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:38.757645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:38.759132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605023f1-c8da-4666-903d-49408d5c2fd2 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:38.768465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 772/1528] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:40.909143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d76c-8cf4-41cb-8822-6575fb24d13e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:40.909690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d76c-8cf4-41cb-8822-6575fb24d13e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:40.909690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5d76c-8cf4-41cb-8822-6575fb24d13e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:40.911110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 757/1529] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:40.916175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468adf3-e2ad-4928-b606-ad704c5a62d8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:40.916613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468adf3-e2ad-4928-b606-ad704c5a62d8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:40.916726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468adf3-e2ad-4928-b606-ad704c5a62d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:40.935590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d468adf3-e2ad-4928-b606-ad704c5a62d8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:40.936777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d468adf3-e2ad-4928-b606-ad704c5a62d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:40.944354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 773/1530] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:41.231912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d76c-8cf4-41cb-8822-6575fb24d13e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:41.242570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5d76c-8cf4-41cb-8822-6575fb24d13e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:41.246145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:41.247276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:41.247550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:41.247814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:41.256542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:41.257949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f2bbf-b913-43d0-b2bf-596ce471a4a0 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:41.267989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 758/1531] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:44.812700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:44.813301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:44.813301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:44.814660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 774/1532] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:44.819710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2f43-5bb7-45a0-8cfd-a180530c638e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:44.820121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2f43-5bb7-45a0-8cfd-a180530c638e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:44.820231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca2f43-5bb7-45a0-8cfd-a180530c638e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:44.839223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-16ca2f43-5bb7-45a0-8cfd-a180530c638e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:44.840923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-16ca2f43-5bb7-45a0-8cfd-a180530c638e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:44.848824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 759/1533] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:45.159221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:45.160883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:45.225890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:45.239031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffcc5-ad07-4e31-a057-7de568b93d33 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:45.242516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:45.243143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:45.243418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:45.243686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:45.252446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:45.253869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cc010-251f-44ee-a29c-a1a81f1d4012 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:45.264016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 775/1534] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:47.419417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:47.419947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:47.419947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:47.421237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 760/1535] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 18 06:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:22:47.426498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30304-19a9-427c-8db7-aa3c0ca369f7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:47.426978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30304-19a9-427c-8db7-aa3c0ca369f7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:47.427084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb30304-19a9-427c-8db7-aa3c0ca369f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:47.448763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9fb30304-19a9-427c-8db7-aa3c0ca369f7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:22:47.449994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9fb30304-19a9-427c-8db7-aa3c0ca369f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:47.458076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 776/1536] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 18 06:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:22:47.789756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:22:47.791456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:22:47.799771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:47.806180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629de29-a21e-4df7-9c99-9d88d1b211ed None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:22:47.809671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:22:47.810255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:22:47.810528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:22:47.810827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:22:47.819584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:22:47.820996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699cfd6-f977-40c7-8c34-415fe2cbfc4a service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:22:47.830176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 761/1537] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:22:57.443632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=11 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:22:57.826141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:10.593597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dca-7b31-4fd1-83d1-6f504e663fa9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.594838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dca-7b31-4fd1-83d1-6f504e663fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.595383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4dca-7b31-4fd1-83d1-6f504e663fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.623133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-40fa4dca-7b31-4fd1-83d1-6f504e663fa9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:10.624405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-40fa4dca-7b31-4fd1-83d1-6f504e663fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:10.635956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 777/1538] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:10.644705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a90c5-8484-4f1e-8eb7-2cf0ac907978 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.645775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a90c5-8484-4f1e-8eb7-2cf0ac907978 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.646219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a90c5-8484-4f1e-8eb7-2cf0ac907978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.663248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.663918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.664351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.684440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-174a90c5-8484-4f1e-8eb7-2cf0ac907978 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:10.688231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-174a90c5-8484-4f1e-8eb7-2cf0ac907978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:10.692401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:10.693928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:10.710675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 778/1539] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:10.712640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 762/1540] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:10.717020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:10.721737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.722028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.722301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.732195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085a4a9-5da3-41f0-a94e-821135a10b03 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:10.743328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:10.744222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.744670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.745104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.745934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:10.746397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:10.748192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:10.767849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:10.769964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:10.770399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 779/1541] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:10.778457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:10.792246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 763/1542] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:10.793334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe9d69-8cc8-4a31-8ed2-8c96f8368075 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:10.797832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:10.802989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:10.803811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.804265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.804702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.805496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:10.806740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:10.806740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:24:10.806740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:24:10.808448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:10.816991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b259c5e-3f1a-495e-9cea-42ca886e50ef None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:10.833154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:10.834166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.834713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.835142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.836400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:10.836881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:10.838076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:10.838076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:24:10.838076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:24:10.839330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b859e-cd42-49b3-a6f7-4c81d70039dd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:10.869943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 764/1543] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:24:10.871683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.872167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.872598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.880452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 780/1544] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:10.881365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:10.907963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:10.910168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:10.990605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf3872-726d-4795-8794-b04a42218e38 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:10.996894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:10.997613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:10.997900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:10.998177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:10.998904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.000896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5649f-223a-4ad5-b27b-b62ad9582e07 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.001566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 765/1545] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.003890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.015481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86662877-52fa-49b2-8ddf-681bedcd3d09 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.020829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.021387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.021708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.022073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.022967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.024130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:11.024130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:24:11.024130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:24:11.025382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582f70b-677e-4c73-9075-598d8ebe3a06 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.034169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 781/1546] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:10 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.038641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5185-e415-470a-a42a-c37b860197e3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.038641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5185-e415-470a-a42a-c37b860197e3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.038641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff5185-e415-470a-a42a-c37b860197e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.049943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 766/1547] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/users => generated 282 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.052277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540b1a-c6c0-4d16-adcb-6ce2f5182f71 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.052277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540b1a-c6c0-4d16-adcb-6ce2f5182f71 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.052277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540b1a-c6c0-4d16-adcb-6ce2f5182f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.065918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d6ff5185-e415-470a-a42a-c37b860197e3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.067286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d6ff5185-e415-470a-a42a-c37b860197e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.071626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8f540b1a-c6c0-4d16-adcb-6ce2f5182f71 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.072807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8f540b1a-c6c0-4d16-adcb-6ce2f5182f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.133815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 782/1548] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.141058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6bb7-8068-4fe5-b1f0-62964408d5b2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.141058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6bb7-8068-4fe5-b1f0-62964408d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.141058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6bb7-8068-4fe5-b1f0-62964408d5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.143322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 767/1549] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.145614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.146115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.146559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.170351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b2ce6bb7-8068-4fe5-b1f0-62964408d5b2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.172014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b2ce6bb7-8068-4fe5-b1f0-62964408d5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.173762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.175763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.184914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 783/1550] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.198819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.198819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.198819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.199654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 768/1551] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.199654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.209726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbf1ec-a0e9-42db-b5b3-02d5ec5f3c38 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.211543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.212768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.214519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.215153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.215434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.215710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.216345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.218973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4f22b-871a-40ed-b9aa-6a30d6da0acc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.222727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 784/1552] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.223593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.234401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778fea3-696e-442b-9369-8a8e15c03585 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.239367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.239957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.240231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.240498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.241188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.242606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 769/1553] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.242908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c9b99c-0a80-43d5-9371-3260166b374a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.243085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808896dd-7200-409f-9876-38cce7cfc987 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.243366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808896dd-7200-409f-9876-38cce7cfc987 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.243694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808896dd-7200-409f-9876-38cce7cfc987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.254621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 785/1554] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:11.255841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff300176-0bde-4e93-9d5d-d7c7a85e1d9f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.256128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff300176-0bde-4e93-9d5d-d7c7a85e1d9f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.256402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff300176-0bde-4e93-9d5d-d7c7a85e1d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.264161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-808896dd-7200-409f-9876-38cce7cfc987 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.265646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:11.266005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-808896dd-7200-409f-9876-38cce7cfc987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.274491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 770/1555] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.275708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f0c9-e5b2-42c5-82ad-1c82e4f49c9a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.275989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f0c9-e5b2-42c5-82ad-1c82e4f49c9a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.276261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87f0c9-e5b2-42c5-82ad-1c82e4f49c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.277664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ff300176-0bde-4e93-9d5d-d7c7a85e1d9f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.279120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ff300176-0bde-4e93-9d5d-d7c7a85e1d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.287162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 786/1556] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.288355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.288638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.288912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.296907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ff87f0c9-e5b2-42c5-82ad-1c82e4f49c9a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.298140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ff87f0c9-e5b2-42c5-82ad-1c82e4f49c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.305929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 771/1557] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.309925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.309925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.309925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.310189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.310189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.318885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 787/1558] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.322205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.328572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.329765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.332392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddff51-f94e-4d6c-9c2f-eb30103ef1e7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.336959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.337545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.337760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 772/1559] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.337896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.338103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.338528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.338753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.339043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.340658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.350508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c3c05-9f1b-4dad-acda-940724d0e821 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.354026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 788/1560] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.354026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.356458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.357050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.357327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.357593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.358260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.359878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.364691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5484f-4c17-4166-9eb3-c565c9abbdb8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.369106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.369684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.372740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 773/1561] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:11.372886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.372886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.373083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.373181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.373181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.377325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:11.383619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270e0d-50c3-4f97-a436-79fc9c934586 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.387531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 789/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:11.387531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.390194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.390818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.391096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.391363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.392003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.392302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.393883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a85a8-e06a-4908-a220-bafc2eea02a0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.397004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92866b55-ea9b-40d0-bdc2-443d1fab9181 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.401401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.401974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.402248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.402515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.403171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.403465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.405064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.413139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 774/1563] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.413139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.413139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.413139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.418615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 790/1564] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.418615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.427516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.428739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.429861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3563ff9-fd2c-4160-b0f6-32b0e3c1d62e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.434444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.435059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.435339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.435613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.436911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.437302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 775/1565] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.437421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.438057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.438820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.447366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357f68b-956e-4129-a6e4-c7f483428641 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.452445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.453034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.453308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.453577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.454212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.454509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.462978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 791/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.463139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.463245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.469251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 776/1567] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.470142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.472429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa96e0-02de-46f2-8e25-8cb1b7384c24 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.477801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.478402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.478710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.478977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.479608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.479749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a45ce-a697-48e7-8d71-a8acf82dcf85 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.481497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.484279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.484852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.485126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.485394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.486020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.487601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.499619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 777/1568] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:11.500638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.512244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ae9d5-58c0-4a27-88b2-b9f4d58db8ed admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.517102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 792/1569] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.517102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.578003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.579014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.579462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.579896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] PATH_INFO: `/v3/projects/0ea7ce44c34647ef8cd69334c88d13ee/users/18bc977aa32042229ac91a7dce501836/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.585119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b8bee-3e81-4be9-80ae-b4ce4c5a334d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.592197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.592835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.593249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.593380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.593995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.595941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.596925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea7ce44c34647ef8cd69334c88d13ee, user_id=18bc977aa32042229ac91a7dce501836, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.599606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.612136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 778/1570] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:11 2022] PUT /identity/v3/projects/0ea7ce44c34647ef8cd69334c88d13ee/users/18bc977aa32042229ac91a7dce501836/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:11.613021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.623216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 793/1571] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.624579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.678876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ff897-ac4e-4302-9cd9-d7250df0e607 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.688009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98833d5-baf7-46d4-9a3b-95a7f9d4eca9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.692707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.693302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.693576 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.693838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.694474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:11.694800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.696396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.703303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:11.710731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 794/1572] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:11.712598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.722907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:11.723958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7898429-22e5-4ab7-8932-464d6152b23c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.729669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.730471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.730842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.731121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.731753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.734233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.736044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.736641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.736919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.737187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.737808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.739826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.761725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 795/1573] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.762629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.770499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 779/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.770499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.828041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26caec-2678-4004-a4c6-0a0bf2ac1770 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.830419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b734a-6d4e-4d71-a88d-9c6622c555f7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.832784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.833396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.833679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.833954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.834604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.835570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.836218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.836489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.836629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.836740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] PATH_INFO: `/v3/projects/91ef3478fd574180abe99393ba769124/users/7a6b8fbd31464e4d99449c74b4d1dda5/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.851835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ef3478fd574180abe99393ba769124, user_id=7a6b8fbd31464e4d99449c74b4d1dda5, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.854476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746444a-ff07-4f37-9725-5c3148776d6e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.864744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 796/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.865597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.871111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 780/1576] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:11 2022] PUT /identity/v3/projects/91ef3478fd574180abe99393ba769124/users/7a6b8fbd31464e4d99449c74b4d1dda5/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:11.872444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.872828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.873094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.901039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] MFA Rules not processed for user `18bc977aa32042229ac91a7dce501836`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:11.902407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:11.930809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0641167-1f8c-4f6d-ba1e-5d30efd2b238 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.935710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.936733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.937015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.937285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.937910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.940076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.960349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 797/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:11.961137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.974344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e61513-b718-45e5-9922-dd6abe0c57f4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.975930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 781/1578] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:11.979623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:11.980797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.981387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.981669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.981938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.982557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:11.985003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:11.992601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542cf6-a7db-4aa0-b540-3145108b07e6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:11.997230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:11.997833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:11.998249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:11.998631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:11.999313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.001464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.006847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 798/1579] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.007758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.017332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d4f4-7ba0-4945-a4a9-e3c70c665a6d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.021831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.022500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.022888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.023192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.023990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.026155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.035914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 782/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:11 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.035914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.047030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 799/1581] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/users => generated 283 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.047843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.059098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e7e76-d96e-4fb2-9f92-7b0a5268d0e3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.062756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.072336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.099760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbc831-1c1d-49eb-8cef-1a06674efd71 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.106795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.107406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.108856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.108856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.108856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.110520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bafad-a004-452b-962f-d00553bc7dfe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.124864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.125596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.125945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.126281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.127047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.129447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.135666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 783/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.136922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.137222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.137578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.156139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 800/1583] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.156972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.157756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:12.159218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:12.166317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50137cb-e497-4acd-a66e-a79fd0eb97cd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.166842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 784/1584] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:12.167603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.172662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.173267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.173555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.173824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.174501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.176134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.178068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d9aed-13f3-416a-900d-9efe690a7b0c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.182847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.183442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.183716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.183989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.184612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.187023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 801/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.187145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.187833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.197361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0f95a-3b0b-4394-a35b-4bdab7de26e8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.202544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.203152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.203436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.203711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.204332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.206102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.218742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 785/1586] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.218924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 802/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.219048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.219135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.228426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2a67-680c-4489-b268-90906a486e32 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.280497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704898fd-e0b4-4a10-ac2f-c0bd52e03c8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.286010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.286625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.286907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.287179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.287794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.288564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.289267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.289656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.289754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.290067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.290804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.292425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.302385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 803/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.303436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.314154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 786/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.314154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.315096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc5b78-0587-43c8-a9dd-037e7fe6febd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.319795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.320550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.320918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.321270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.322073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.322221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165d9a-dda6-418c-b39e-4844e78bffe9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.323916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.327416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.327993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.328268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.328537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.329199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.330820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.336674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 804/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.336674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.342130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 787/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.343318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.352865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a9845-debd-4eed-b183-3801bffe80bd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.357786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.359031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.359305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.359571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.360261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.361813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.371957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 788/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.371957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.382353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb912d-2a9c-4e78-a6ca-ecb581934d28 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.387819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.388388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.388666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.388929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.390274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.392202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.396156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1c4a3-db20-4247-a2ed-6ba4fbb5ddd8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.399748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.400350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.400627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.400923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.411383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.412818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.413921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 789/1593] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:12.414805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.423025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 805/1594] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:12.423756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.424515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f17e-113f-4c6d-9069-0d108736ee8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.429159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.429736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.430008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.430315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.430936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.432507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.432788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14693716-3282-47cf-a99f-0b33d70b55d6 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.437465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.438160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.438392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.438710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.439335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.440911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.443476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 790/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.444367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.448028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.456521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 806/1596] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.457640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.460970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977ff0-41d1-414d-b16e-f82c5bf0e84d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.467129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.468840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.468840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.469110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.469845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.471741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.477527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e0899-2de0-4f54-bead-f1b6dd7a20fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.484857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 791/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.488128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.488306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.488918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.489197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.489478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.490113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.491729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.499761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22d7d1-cbf3-4d51-a439-2e6b1fcb5b89 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.503661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 807/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.504458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.507431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.508039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.508316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.508587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.510086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.511860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.519421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d944237-c721-4614-9839-7f5310d3bbe7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.523928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 792/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.525862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.526770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.526980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.527353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] PATH_INFO: `/v3/projects/0c5d2a3f2d944529b8c505fa23b5a10f/users/d85160b01d824bc3a32b425fab98d2c7/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.527653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.534755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.539117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba129d-c27d-4bbc-a554-bcdcd30fa721 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.546215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5d2a3f2d944529b8c505fa23b5a10f, user_id=d85160b01d824bc3a32b425fab98d2c7, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.547170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.547367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:12.547367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:24:12.547367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:24:12.548538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.548932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.549268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.549374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.549981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.551702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.562916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 793/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.563565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.568906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 808/1601] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/0c5d2a3f2d944529b8c505fa23b5a10f/users/d85160b01d824bc3a32b425fab98d2c7/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.569934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.574695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a0d12-e2b4-43b0-b1e3-21851110f4d4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.646473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.647237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.648316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.648316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] PATH_INFO: `/v3/projects/9cbc859294e9456eb2ca751279a17f52/users/6c945a3e6a4946559aeadc63566d29f4/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.658380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa3511-defe-4ce2-ae06-56531c9d0a75 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.663638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.664334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.664615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.664888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] PATH_INFO: `/v3/projects/6ae79386aea24d858d7e8de0a4ae27b3/users/b12c8868fa4e4fe1b12591528cdb926a/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.665859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cbc859294e9456eb2ca751279a17f52, user_id=6c945a3e6a4946559aeadc63566d29f4, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.669612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.682613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae79386aea24d858d7e8de0a4ae27b3, user_id=b12c8868fa4e4fe1b12591528cdb926a, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.685711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.686884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 794/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/9cbc859294e9456eb2ca751279a17f52/users/6c945a3e6a4946559aeadc63566d29f4/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.687739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.704450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 809/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/6ae79386aea24d858d7e8de0a4ae27b3/users/b12c8868fa4e4fe1b12591528cdb926a/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.705389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.761051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb6e5-7b5d-4275-a544-909d3322cce4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.782070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f2a38-1dc6-4d2b-bc87-099525ce323e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.787640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.788393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.788749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.789085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] PATH_INFO: `/v3/projects/dade588a399b4216aa97e0f86e509ba3/users/85bf4e10a7674509961e17a3ed345f9b/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.810851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dade588a399b4216aa97e0f86e509ba3, user_id=85bf4e10a7674509961e17a3ed345f9b, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.814374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.838656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 810/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/dade588a399b4216aa97e0f86e509ba3/users/85bf4e10a7674509961e17a3ed345f9b/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.839988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.853126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.853954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.854372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.854834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.855778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.859280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.870917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 795/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:12.871964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:12.882876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21e12d-4649-444b-bb60-5010704034b6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.888954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.898011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:12.924723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a56bbd-821e-4b75-b86b-54c6ab6c1641 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:12.929409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.930142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.930489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.930858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] PATH_INFO: `/v3/projects/22ba57c2ca3e4ff9a4b77d8cecbfa4e9/users/93d23721dfd34020b659707b65eb2c33/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.946444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ba57c2ca3e4ff9a4b77d8cecbfa4e9, user_id=93d23721dfd34020b659707b65eb2c33, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.947726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:12.948462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.948814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.949147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fef75-1f49-4e2e-a591-7f8f200a4f40 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.949282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] PATH_INFO: `/v3/projects/80a95e5ef49a45b88fd71708a1702667/users/cb9c37a2a7a2491f81cd1128e2ed6a69/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.965419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a95e5ef49a45b88fd71708a1702667, user_id=cb9c37a2a7a2491f81cd1128e2ed6a69, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:12.965685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 811/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/22ba57c2ca3e4ff9a4b77d8cecbfa4e9/users/93d23721dfd34020b659707b65eb2c33/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.966947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e043b0-967e-4502-a1b5-43c6bef6a017 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.967796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e043b0-967e-4502-a1b5-43c6bef6a017 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.968078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e043b0-967e-4502-a1b5-43c6bef6a017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.969136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a85f8-99a2-42fa-88c6-2899369dac13 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:12.985100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 796/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:12 2022] PUT /identity/v3/projects/80a95e5ef49a45b88fd71708a1702667/users/cb9c37a2a7a2491f81cd1128e2ed6a69/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:12.986517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70edd37-7698-461b-bbda-b2123e44b412 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:12.986933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70edd37-7698-461b-bbda-b2123e44b412 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:12.987306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70edd37-7698-461b-bbda-b2123e44b412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:12.993246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-15e043b0-967e-4502-a1b5-43c6bef6a017 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:12.994642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-15e043b0-967e-4502-a1b5-43c6bef6a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.007290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d70edd37-7698-461b-bbda-b2123e44b412 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.008845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d70edd37-7698-461b-bbda-b2123e44b412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.064947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 812/1608] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.066109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0c92-add4-4817-8eda-d87088cb9d0d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.066394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0c92-add4-4817-8eda-d87088cb9d0d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.066700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0c92-add4-4817-8eda-d87088cb9d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.079506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 797/1609] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.080660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.080944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.081218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.109420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d84b0c92-add4-4817-8eda-d87088cb9d0d None None] MFA Rules not processed for user `b12c8868fa4e4fe1b12591528cdb926a`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.111273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d84b0c92-add4-4817-8eda-d87088cb9d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.113118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] MFA Rules not processed for user `85bf4e10a7674509961e17a3ed345f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.114363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.190981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 798/1610] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.195694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.203270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48821d94-02fe-405a-8b8c-2b8aeb5b4e2c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.208373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 813/1611] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.208542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.209227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.209564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.209894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] PATH_INFO: `/v3/projects/d81b5577d70b4f55b0a48b7368814468/users/1ad67acd84524ffc915c6105b0fe533d/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.210660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.211113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.211535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.239542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81b5577d70b4f55b0a48b7368814468, user_id=1ad67acd84524ffc915c6105b0fe533d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.242851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.244664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.246687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.263081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 814/1612] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.264567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.265051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 799/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:13 2022] PUT /identity/v3/projects/d81b5577d70b4f55b0a48b7368814468/users/1ad67acd84524ffc915c6105b0fe533d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:13.265937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.280291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.281582 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db57ab-911b-441e-844a-8efbce02ee3b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.319519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.346880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c761c6-a776-4771-9c8c-01b509e56cbd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.351282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.351647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.351928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.352202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.359740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.360694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.361135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.361581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.362888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.365485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.379874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 815/1614] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:13.379874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.390521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28085694-0729-4fb6-8667-fa76c0f2a4d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.395512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.397154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.397801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.397801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.418279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.419864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.432612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.432863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 816/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:13.433669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.434872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6a033-3b29-4e6a-b8e7-8ec98c692704 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.460338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 800/1616] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:13.465493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde483ca-065a-48d9-ad98-da791bd01ea8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.465913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde483ca-065a-48d9-ad98-da791bd01ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.466064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde483ca-065a-48d9-ad98-da791bd01ea8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.473166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 801/1617] 10.222.0.20 () {54 vars in 965 bytes} [Sun Sep 18 06:24:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:13.474073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde483ca-065a-48d9-ad98-da791bd01ea8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.491572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde483ca-065a-48d9-ad98-da791bd01ea8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.496718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.497548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.497828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.498098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.512596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.514229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6926e0d7-fbd3-4282-ace1-bc65047de466 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.514613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024dbfa-1a43-4527-b531-25d4a9921371 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.519505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.520178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.520521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.520803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.528650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 802/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:13.530886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4140-fc6a-4f0b-b8de-931919283bc5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.531172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4140-fc6a-4f0b-b8de-931919283bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.531733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4140-fc6a-4f0b-b8de-931919283bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.567902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-00bb4140-fc6a-4f0b-b8de-931919283bc5 None None] MFA Rules not processed for user `1ad67acd84524ffc915c6105b0fe533d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.569324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-00bb4140-fc6a-4f0b-b8de-931919283bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.601156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.602829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.631634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 817/1619] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:13.634827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.662912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f3171-8585-4a2e-9538-67e0b1fae0d5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.666970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 803/1620] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.668463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4eb2-810d-4ce2-9859-885e88097799 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.668787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4eb2-810d-4ce2-9859-885e88097799 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.669012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4eb2-810d-4ce2-9859-885e88097799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.673277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.674142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.674725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.675071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] PATH_INFO: `/v3/projects/80a95e5ef49a45b88fd71708a1702667/users/cb9c37a2a7a2491f81cd1128e2ed6a69/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.679626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a95e5ef49a45b88fd71708a1702667, user_id=cb9c37a2a7a2491f81cd1128e2ed6a69, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.683348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.700329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 818/1621] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:13 2022] PUT /identity/v3/projects/80a95e5ef49a45b88fd71708a1702667/users/cb9c37a2a7a2491f81cd1128e2ed6a69/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:13.701265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.731231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-44cb4eb2-810d-4ce2-9859-885e88097799 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.732795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44cb4eb2-810d-4ce2-9859-885e88097799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:13.761703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.826803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be68f-1320-4dc4-bdeb-3c0eb59b3c60 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:13.840537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:13.841232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.843761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.843915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.845737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:13.853941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:13.862477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:13.868725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 804/1622] 10.222.0.20 () {58 vars in 1062 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:13.871328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:13.871775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:13.872230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:13.909328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 819/1623] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/projects => generated 373 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:13.913152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:13.923956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] MFA Rules not processed for user `cb9c37a2a7a2491f81cd1128e2ed6a69`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:13.925486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:14.024320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 805/1624] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:14.024829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.035804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb319e6-bca0-41a0-94ac-4ad30a51b057 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.046942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.047947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.049191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.049939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.167335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3c49-eca5-498f-9a52-963d729aedec None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.175134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.175773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.176118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.176533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.178163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.180613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.199066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.202241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.216536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 820/1625] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:14.216793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 806/1626] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:14 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:14.218161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.221298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.251346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b32e11-c634-4ab0-bbad-3b9b29ae1e52 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.270108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.270920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.271209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.271486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.272155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.276348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.300000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 807/1627] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:14.303555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.307024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638007c4-d5fe-4718-a930-6f5fbab164df service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.315827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad9336-4807-445a-a2e2-74354f412b37 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.317036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.319333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.320748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.320991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.321210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.321776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.322058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.322375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] PATH_INFO: `/v3/projects/83a714b488774bc5a96530181a69e73a/users/20db2aecaee04f8d87a600aa76b3bf10/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.357080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.362667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.364978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.375141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a714b488774bc5a96530181a69e73a, user_id=20db2aecaee04f8d87a600aa76b3bf10, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.389863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.437959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 821/1628] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:14.453987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 808/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:14 2022] PUT /identity/v3/projects/83a714b488774bc5a96530181a69e73a/users/20db2aecaee04f8d87a600aa76b3bf10/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:14.457004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.479897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d1a33-01d0-4b1c-bb4b-1a5477cb05b6 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.544431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:14.625344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.664353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.672125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.672507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.672783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.678091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.686290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.732655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 822/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:14.733751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:14.759470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5e883-46fc-412c-b81b-aabee1dd0649 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.773390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.774886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.775567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.776124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] PATH_INFO: `/v3/projects/83a714b488774bc5a96530181a69e73a/users/20db2aecaee04f8d87a600aa76b3bf10/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.781835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83a714b488774bc5a96530181a69e73a, user_id=20db2aecaee04f8d87a600aa76b3bf10, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.785620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab9768-59b3-42a2-8240-d69fa11b45b4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.791824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb70f4-a055-43ac-8b59-f94cb2f9fd76 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:14.806395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:14.807056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.807339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.807713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.808408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 823/1631] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:14 2022] PUT /identity/v3/projects/83a714b488774bc5a96530181a69e73a/users/20db2aecaee04f8d87a600aa76b3bf10/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:14.818804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e31e2-d329-4fc7-840d-17930d9d2d2c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:14.819264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e31e2-d329-4fc7-840d-17930d9d2d2c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:14.819696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e31e2-d329-4fc7-840d-17930d9d2d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:14.875910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2d4e31e2-d329-4fc7-840d-17930d9d2d2c None None] MFA Rules not processed for user `20db2aecaee04f8d87a600aa76b3bf10`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:14.877620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2d4e31e2-d329-4fc7-840d-17930d9d2d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:14.955306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:14.956815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:14.987884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 809/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:15.034439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 824/1633] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:15.102020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:15.102020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea9a0-cec5-4e3c-932f-8ee21d2a8784 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8321-3456-47c6-8766-1155f8c93fe3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8321-3456-47c6-8766-1155f8c93fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8321-3456-47c6-8766-1155f8c93fe3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 825/1634] 10.222.0.20 () {54 vars in 965 bytes} [Sun Sep 18 06:24:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32569c-70a1-47aa-ab5a-8a9573e368de None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.119219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32569c-70a1-47aa-ab5a-8a9573e368de None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.119746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32569c-70a1-47aa-ab5a-8a9573e368de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.188127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5d32569c-70a1-47aa-ab5a-8a9573e368de None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:15.189384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5d32569c-70a1-47aa-ab5a-8a9573e368de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:15.279443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:15.280455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.280976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.281453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.315770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:15.317282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad2e7-658f-4ded-8d06-ca7350ea018b service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:15.339197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 810/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:15.387633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 826/1636] 10.222.0.20 () {58 vars in 1062 bytes} [Sun Sep 18 06:24:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:15.404065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a5ecd-dea1-4bce-bd55-3cb71ea90bba None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.404360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a5ecd-dea1-4bce-bd55-3cb71ea90bba None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.404686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a5ecd-dea1-4bce-bd55-3cb71ea90bba None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.408013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 811/1637] 10.222.0.20 () {54 vars in 912 bytes} [Sun Sep 18 06:24:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:15.413399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.413890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.414343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.443721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:15.445109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:15.462907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 827/1638] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:24:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:15.576177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a5ecd-dea1-4bce-bd55-3cb71ea90bba None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:15.596189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a5ecd-dea1-4bce-bd55-3cb71ea90bba None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:15.601806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:15.603196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:15.603827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.604102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.605009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.613779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123c9ce-4b4a-445f-bf43-59eb90101643 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:15.618330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:15.620138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea1d88-f23a-4aa9-a750-4cdbfb9fdc64 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:15.627495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:15.628115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.628400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.628651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.635982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 812/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:15.659292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:15.793012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f5d7-48bd-46f2-9f6c-c045b43c92bb None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.793593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f5d7-48bd-46f2-9f6c-c045b43c92bb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.793593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2f5d7-48bd-46f2-9f6c-c045b43c92bb None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.795895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 813/1640] 10.222.0.20 () {54 vars in 912 bytes} [Sun Sep 18 06:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:24:15.803327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:15.803609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:15.803868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:15.813012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:15.815220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:15.848408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:15.850141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:15.850269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 828/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:15.862894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 814/1642] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:24:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:15.996899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:16.011355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6f607-8558-466d-aebd-dc633c25d3ad service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:16.016760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:16.018525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:16.019030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:16.019510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:16.036946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:16.039060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a736c-1b24-480e-a07b-71f3f516f25b service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:16.070811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 829/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:16.215982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:16.272483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:16.318153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d24ac-cb33-415d-94cc-5567afcebefc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:16.323435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:16.324084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:16.324504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:16.324828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] PATH_INFO: `/v3/users/85bf4e10a7674509961e17a3ed345f9b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:16.326156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=85bf4e10a7674509961e17a3ed345f9b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:16.329075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:16.427400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85bf4e10a7674509961e17a3ed345f9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:16.442120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85bf4e10a7674509961e17a3ed345f9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:16.443085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85bf4e10a7674509961e17a3ed345f9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:16.443818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85bf4e10a7674509961e17a3ed345f9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:16.464044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85bf4e10a7674509961e17a3ed345f9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:16.506805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 815/1644] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:16 2022] DELETE /identity/v3/users/85bf4e10a7674509961e17a3ed345f9b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:16.511133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:16.512016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:16.512515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:16.543250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:16.545787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:16.707837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 830/1645] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:16.766784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:16.766784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:16.933163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680fb73-3f88-45a8-bd1d-413a8fc32edb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:16.940185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:16.941249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:16.941799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:16.942215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:16.955479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:16.958526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9c05c-0b8e-4b33-95d5-264c9c30b935 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:16.974568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 816/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:18.411599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:18.413927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:18.437832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a74bc-4496-4a9e-bd6d-c2e054884c70 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:18.453532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:18.453532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:18.453532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:18.453532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] PATH_INFO: `/v3/projects/dade588a399b4216aa97e0f86e509ba3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:18.467217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dade588a399b4216aa97e0f86e509ba3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:18.469599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:18.562806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dade588a399b4216aa97e0f86e509ba3', 'request_id': 'req-da939419-1e18-40f7-abf7-6c039cfe541d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:18.576889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dade588a399b4216aa97e0f86e509ba3', 'request_id': 'req-da939419-1e18-40f7-abf7-6c039cfe541d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:18.595383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dade588a399b4216aa97e0f86e509ba3', 'request_id': 'req-da939419-1e18-40f7-abf7-6c039cfe541d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:18.616043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] The token cache is being invalidate because project dade588a399b4216aa97e0f86e509ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:18.616823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:18.617818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-da939419-1e18-40f7-abf7-6c039cfe541d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:18.622351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 831/1647] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:18 2022] DELETE /identity/v3/projects/dade588a399b4216aa97e0f86e509ba3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:18.627684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:18.628180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:18.628278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:18.667874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:18.671135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:18.778611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:18.807517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 817/1648] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:18.808392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:18.808678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:18.808952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:18.851061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:18.853104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:18.871787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:18.882111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 832/1649] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:18.886368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81eea3c-d121-4115-b692-bc6d40887b1c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:18.893374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:18.894230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:18.894681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:18.895226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:18.896099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:18.896579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:18.898385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:18.917669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 818/1650] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:18.919584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:18.931795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7996d05-fd98-45ca-9c27-e9c73cf2c40c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:18.939641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:18.940691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:18.941890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:18.942318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:18.943365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:18.945650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:18.980815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:19.000585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.001883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 833/1651] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:19.017002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31f294-f10e-4618-b3ac-c8b188096beb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.130367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.131006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.132552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.132836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.133479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.135728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.185989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 819/1652] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:18 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:19.191837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.206945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db57fa3-6f8f-4346-a604-9edbcc910500 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.220166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.220804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.221428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.221712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.226083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.229191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.249825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.258365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 834/1653] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:19.266106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b3b2-18a5-42c5-8489-d5783ebf088b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.284247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.285321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.285734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.286174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.287282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.288980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0756f-1771-489d-b7d1-9d6964259e2f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.315059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.317877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 820/1654] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:19.328530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4febee7-6866-4dde-bddb-68b0623bfbf5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.337817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.338513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.338816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.339635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] PATH_INFO: `/v3/projects/d3cfb3ce1c4f48dd95d67b855a09db98/users/058ff9c4fc9847508c5025c419eb0a5c/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.367000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cfb3ce1c4f48dd95d67b855a09db98, user_id=058ff9c4fc9847508c5025c419eb0a5c, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.370130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.405867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 835/1655] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:19 2022] PUT /identity/v3/projects/d3cfb3ce1c4f48dd95d67b855a09db98/users/058ff9c4fc9847508c5025c419eb0a5c/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:19.418267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.418607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.418847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.467938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] MFA Rules not processed for user `058ff9c4fc9847508c5025c419eb0a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:19.469189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:19.625205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.627637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:19.636889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 821/1656] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:19.645561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:19.766501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:19.813924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63d2a5-3f31-43ea-9fad-09693c43a838 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.823048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84d50b-6d71-4afa-8ccf-468dca195255 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:19.826478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.828465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.828885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.829157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.829927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.836073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.838992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:19.839391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:19.839760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:19.840101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] PATH_INFO: `/v3/users/18bc977aa32042229ac91a7dce501836` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:19.844700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bc977aa32042229ac91a7dce501836)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:19.847879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:19.847879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Sep 18 06:24:19.847879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: Sep 18 06:24:19.850430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:19.907601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 822/1657] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:19 2022] POST /identity/v3/projects => generated 373 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:19.908862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:20.056841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bc977aa32042229ac91a7dce501836'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:20.093314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bc977aa32042229ac91a7dce501836'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:20.095034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bc977aa32042229ac91a7dce501836'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:20.095844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bc977aa32042229ac91a7dce501836'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:20.104487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82c67a-3051-42e3-909a-ba327a5f046f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:20.118462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:20.119293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.119718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.120130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.120960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:20.124727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:20.143228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-881246f7-fded-4cf8-93a6-ec8720810cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bc977aa32042229ac91a7dce501836'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:20.176124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 823/1658] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:19 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:20.180041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:20.186065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:20.189682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 836/1659] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:19 2022] DELETE /identity/v3/users/18bc977aa32042229ac91a7dce501836 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:20.195040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.197479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.197479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.220561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9340e-427d-4345-a62a-0fe0f154510e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:20.251872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:20.253331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:20.361082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:20.409269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 837/1660] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:20.412635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:20.414416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.415839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.416436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.417960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:20.421745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:20.437997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 824/1661] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:20.441968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:20.468265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4405e94-8103-496c-9992-4a80f95b1292 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:20.479795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:20.480634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.480977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.481557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] PATH_INFO: `/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782/users/6c23ec3513d04d91b1b46a01c5297a50/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.520242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a144d0deafe4c0baf0c58e0cfc0b782, user_id=6c23ec3513d04d91b1b46a01c5297a50, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:20.523081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f104d0-836f-47ad-acc5-3ab318bf0e97 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:20.551958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 838/1662] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:20 2022] PUT /identity/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782/users/6c23ec3513d04d91b1b46a01c5297a50/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:20.553275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:20.573912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b29ea-7b0f-481a-848a-dd67ab9d988e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:20.579065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8605b-fb7a-49b2-972e-3ce247337edd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.579351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8605b-fb7a-49b2-972e-3ce247337edd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.579669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8605b-fb7a-49b2-972e-3ce247337edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.668291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8ab8605b-fb7a-49b2-972e-3ce247337edd None None] MFA Rules not processed for user `7a6b8fbd31464e4d99449c74b4d1dda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:20.669534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8ab8605b-fb7a-49b2-972e-3ce247337edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:20.760453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:20.772886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:20.773767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.774128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.774475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.775297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:20.777976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:20.793229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 825/1663] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:20.799091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:20.813493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09b778-4efb-4bf5-ab3a-c48f98be7739 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:20.828409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:20.829321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.829746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.830101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] PATH_INFO: `/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782/users/6c23ec3513d04d91b1b46a01c5297a50/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.836616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a144d0deafe4c0baf0c58e0cfc0b782, user_id=6c23ec3513d04d91b1b46a01c5297a50, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:20.842616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc566-7b95-4770-b701-893ca6eabdc2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:20.843365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 839/1664] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:20.852895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf71fba-98a5-4c69-9dbb-1fa23cfcf2f7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.852895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf71fba-98a5-4c69-9dbb-1fa23cfcf2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.852895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf71fba-98a5-4c69-9dbb-1fa23cfcf2f7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.852895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 840/1665] 10.222.0.20 () {54 vars in 963 bytes} [Sun Sep 18 06:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:20.853718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb83b-36d9-4794-9465-d9874d55bff1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.853998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb83b-36d9-4794-9465-d9874d55bff1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.854266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fb83b-36d9-4794-9465-d9874d55bff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.878433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 826/1666] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:20 2022] PUT /identity/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782/users/6c23ec3513d04d91b1b46a01c5297a50/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:20.891942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:20.891942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:20.892252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:20.904099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-556fb83b-36d9-4794-9465-d9874d55bff1 None None] MFA Rules not processed for user `7a6b8fbd31464e4d99449c74b4d1dda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:20.905581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-556fb83b-36d9-4794-9465-d9874d55bff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:21.047126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] MFA Rules not processed for user `6c23ec3513d04d91b1b46a01c5297a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:21.052374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:21.080246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 841/1667] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:21.086932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:21.087395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:21.088293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:21.186165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:21.188527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:21.232883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 827/1668] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:21.234000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:21.235493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:21.253264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:21.276356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 842/1669] 10.222.0.20 () {58 vars in 1060 bytes} [Sun Sep 18 06:24:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:21.277412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:21.279134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:21.402002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfb56e-dd83-4053-9e09-10d71f32fd00 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:21.406772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:21.407827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:21.408282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:21.408548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:21.413279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672a9f-e978-4eea-b0bf-465e52e51d00 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:21.426261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:21.427537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:21.427971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:21.428382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:21.432022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:21.433549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:21.452016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:21.453904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:21.472784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 843/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:21.479746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 828/1671] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:22.111329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:22.129617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141595-c359-4619-8b98-02cedeeba1b3 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:22.141740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:22.142537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:22.142942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:22.143333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:22.372056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:22.375692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d0a74-a956-467a-a9e3-7f6072452e68 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:22.416529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 844/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:22.498793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:22.518066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9800028-0c8d-4576-86e8-6098f6ac5e73 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:22.532680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:22.533338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:22.533744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:22.533870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] PATH_INFO: `/v3/users/058ff9c4fc9847508c5025c419eb0a5c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:22.535071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=058ff9c4fc9847508c5025c419eb0a5c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:22.537352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:22.641730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:22.644202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058ff9c4fc9847508c5025c419eb0a5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:22.689539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058ff9c4fc9847508c5025c419eb0a5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:22.690419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058ff9c4fc9847508c5025c419eb0a5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:22.691286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058ff9c4fc9847508c5025c419eb0a5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:22.707961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05c02b58-6226-4883-b7ed-d31a0436ce5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058ff9c4fc9847508c5025c419eb0a5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:22.762094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 829/1673] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:22 2022] DELETE /identity/v3/users/058ff9c4fc9847508c5025c419eb0a5c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:22.766563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:22.766883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:22.767727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:22.825837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:22.827285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:22.886159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176db-bdf2-4091-8c43-adba45fcf4fa None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:22.886440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176db-bdf2-4091-8c43-adba45fcf4fa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:22.886742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29176db-bdf2-4091-8c43-adba45fcf4fa None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:22.888388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 830/1674] 10.222.0.20 () {54 vars in 912 bytes} [Sun Sep 18 06:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:24:22.903789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:22.904065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:22.905116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:22.939344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:22.940610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:23.004042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 845/1675] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:23.016440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:23.019798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:23.065059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 831/1676] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:24:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:23.080385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:23.082123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:23.097083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795f5a6-b8cd-43e9-a323-b01816c91906 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:23.097313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:23.102390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:23.102804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:23.103085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:23.103491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:23.122272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:23.124677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:23.135341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca3a4a-7bde-49df-af47-f800f5e5380f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:23.145515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:23.146521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:23.146951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:23.147511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:23.160338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 832/1677] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:23.195734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:23.197389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368d69f-456f-463b-b858-c32fd4ae1c9d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:23.226844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 846/1678] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:23.937281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:23.941176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.000718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3123c78-08a0-477b-a11f-2928211218b7 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:24.042363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:24.043288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.046020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.046453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] PATH_INFO: `/v3/projects/d3cfb3ce1c4f48dd95d67b855a09db98` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.048185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cfb3ce1c4f48dd95d67b855a09db98)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:24.049521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 18 06:24:24.049521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Sep 18 06:24:24.049521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: Sep 18 06:24:24.055566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:24.242292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997e92-e622-4eab-85a2-3adceacf93dc None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.243115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997e92-e622-4eab-85a2-3adceacf93dc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.243386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6997e92-e622-4eab-85a2-3adceacf93dc None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.245699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 847/1679] 10.222.0.20 () {54 vars in 961 bytes} [Sun Sep 18 06:24:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:24.252710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.253229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.253728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.259657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cfb3ce1c4f48dd95d67b855a09db98', 'request_id': 'req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.309338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cfb3ce1c4f48dd95d67b855a09db98', 'request_id': 'req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.315597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:24.319728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.332002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cfb3ce1c4f48dd95d67b855a09db98', 'request_id': 'req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.356797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] The token cache is being invalidate because project d3cfb3ce1c4f48dd95d67b855a09db98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:24.357154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.360238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8bff8503-80ea-4bed-bc27-dbcd5cb75a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.365537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 833/1680] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:23 2022] DELETE /identity/v3/projects/d3cfb3ce1c4f48dd95d67b855a09db98 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:24.368675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.369536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.369822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.416789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] MFA Rules not processed for user `7a6b8fbd31464e4d99449c74b4d1dda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:24.417931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.517595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:24.568960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 848/1681] 10.222.0.20 () {58 vars in 1058 bytes} [Sun Sep 18 06:24:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:24.570606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:24.572491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.602133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 834/1682] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:24.603173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:24.640767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc2640-9643-410c-8a0e-7aceb3588437 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:24.657697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:24.658325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.658601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.658886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.672720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.729139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:24.731061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:24.753061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e468ed-356f-4ed6-a250-b81cea124ced None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:24.765119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 835/1683] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:24.767670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:24.768329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.768611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.769395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] PATH_INFO: `/v3/users/6c23ec3513d04d91b1b46a01c5297a50` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.776584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c23ec3513d04d91b1b46a01c5297a50)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:24.780729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:24.915655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c23ec3513d04d91b1b46a01c5297a50'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.924500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:24.927051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:24.942193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c23ec3513d04d91b1b46a01c5297a50'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.942866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c23ec3513d04d91b1b46a01c5297a50'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.943707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c23ec3513d04d91b1b46a01c5297a50'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.955276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3fe24-212e-43b0-a846-63be1e1ab153 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:24.975307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:24.975953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:24.976236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:24.976548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] PATH_INFO: `/v3/projects/0ea7ce44c34647ef8cd69334c88d13ee` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:24.984311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:24.986218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c23ec3513d04d91b1b46a01c5297a50'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:24.987461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea7ce44c34647ef8cd69334c88d13ee)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:24.989417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:25.017390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 849/1684] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:24 2022] DELETE /identity/v3/users/6c23ec3513d04d91b1b46a01c5297a50 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:25.118430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea7ce44c34647ef8cd69334c88d13ee', 'request_id': 'req-89e1bef7-6e86-4c18-b934-2a35db94786d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.132909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea7ce44c34647ef8cd69334c88d13ee', 'request_id': 'req-89e1bef7-6e86-4c18-b934-2a35db94786d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.157213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea7ce44c34647ef8cd69334c88d13ee', 'request_id': 'req-89e1bef7-6e86-4c18-b934-2a35db94786d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.190550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] The token cache is being invalidate because project 0ea7ce44c34647ef8cd69334c88d13ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:25.191127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.192967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.201141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 836/1685] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:24 2022] DELETE /identity/v3/projects/0ea7ce44c34647ef8cd69334c88d13ee => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:25.202916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:25.204822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:25.406144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa79a-0735-4029-a78c-6eb5d274520e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:25.593128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:25.593802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:25.594075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:25.594345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] PATH_INFO: `/v3/users/cb9c37a2a7a2491f81cd1128e2ed6a69` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:25.597640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9c37a2a7a2491f81cd1128e2ed6a69)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:25.602007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:25.713588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9c37a2a7a2491f81cd1128e2ed6a69'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.738157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9c37a2a7a2491f81cd1128e2ed6a69'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.739111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9c37a2a7a2491f81cd1128e2ed6a69'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.739834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9c37a2a7a2491f81cd1128e2ed6a69'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.811647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e953831a-a777-4c99-9e38-b73556dd3d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9c37a2a7a2491f81cd1128e2ed6a69'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:25.849298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 850/1686] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:25 2022] DELETE /identity/v3/users/cb9c37a2a7a2491f81cd1128e2ed6a69 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:26.374907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:26.376656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:26.410424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1bef7-6e86-4c18-b934-2a35db94786d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:26.593789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:26.597675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:26.597962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:26.598240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] PATH_INFO: `/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:26.602725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a144d0deafe4c0baf0c58e0cfc0b782)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:26.607987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:26.719787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a144d0deafe4c0baf0c58e0cfc0b782', 'request_id': 'req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:26.730105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a144d0deafe4c0baf0c58e0cfc0b782', 'request_id': 'req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:26.739999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:26.754940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a144d0deafe4c0baf0c58e0cfc0b782', 'request_id': 'req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:26.776258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] The token cache is being invalidate because project 3a144d0deafe4c0baf0c58e0cfc0b782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:26.777727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:26.780711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:26.784750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 837/1687] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:26 2022] DELETE /identity/v3/projects/3a144d0deafe4c0baf0c58e0cfc0b782 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:26.790775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de571-b1aa-4e2d-b7d3-2a94f30236c2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:26.791439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de571-b1aa-4e2d-b7d3-2a94f30236c2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:26.791439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6de571-b1aa-4e2d-b7d3-2a94f30236c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:26.829953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bc6de571-b1aa-4e2d-b7d3-2a94f30236c2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:26.833409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bc6de571-b1aa-4e2d-b7d3-2a94f30236c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:26.841524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:26.843277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:26.858282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:26.961675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 851/1688] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:26.972290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:26.975587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:26.979180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.000276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229af136-2e0d-4bbf-a0b1-64b6d3f544ab admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:27.013534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:27.014609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.016040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.016331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] PATH_INFO: `/v3/users/b12c8868fa4e4fe1b12591528cdb926a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.020572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12c8868fa4e4fe1b12591528cdb926a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:27.028183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:27.040570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:27.043900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.066371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 852/1689] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:27.067438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:27.071119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.117037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8498a1-e011-47e1-9373-205747b34506 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:27.133343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:27.138676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.138972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.139249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] PATH_INFO: `/v3/projects/80a95e5ef49a45b88fd71708a1702667` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.142750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12c8868fa4e4fe1b12591528cdb926a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.143048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a95e5ef49a45b88fd71708a1702667)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:27.150913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:27.164591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12c8868fa4e4fe1b12591528cdb926a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.168050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12c8868fa4e4fe1b12591528cdb926a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.172175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12c8868fa4e4fe1b12591528cdb926a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.195062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12c8868fa4e4fe1b12591528cdb926a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.246236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 838/1690] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:26 2022] DELETE /identity/v3/users/b12c8868fa4e4fe1b12591528cdb926a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:27.247326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:27.255611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.291357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a95e5ef49a45b88fd71708a1702667', 'request_id': 'req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.313721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a95e5ef49a45b88fd71708a1702667', 'request_id': 'req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.339475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a95e5ef49a45b88fd71708a1702667', 'request_id': 'req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.364876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] The token cache is being invalidate because project 80a95e5ef49a45b88fd71708a1702667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:27.367443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.368502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4eeec33b-ce1c-4201-81e5-d5b9a85ffbf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:27.382170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 853/1691] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:27 2022] DELETE /identity/v3/projects/80a95e5ef49a45b88fd71708a1702667 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:27.382909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b887964-cf3a-4d4c-89ae-a02467c38d01 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.383208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b887964-cf3a-4d4c-89ae-a02467c38d01 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.383468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b887964-cf3a-4d4c-89ae-a02467c38d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.438949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0b887964-cf3a-4d4c-89ae-a02467c38d01 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:27.440524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0b887964-cf3a-4d4c-89ae-a02467c38d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.517096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ccee7-1d7f-4647-94b6-e742d220f0b3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:27.565936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:27.566911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 854/1692] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:27.570171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.571249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.571550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.628763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:27.630181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.671437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 855/1693] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:27.671437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:27.672559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.756846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:27.757593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.757973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.759719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.762438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:27.763934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:27.767841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c1489-5b7d-403d-a0a4-583d9850ac03 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:27.795551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:27.814459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 839/1694] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 568 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:27.820719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.822071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.822684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.886146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:27.893376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.897626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b15cd6-3998-483f-bda8-e99b1056933d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:27.911307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:27.912225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.917095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 840/1695] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:27.919633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:27.920825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.921045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.924372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:27.945185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:27.947444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:27.968248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9541a6-c6de-4f62-82ff-6e9f812efaa3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:27.985641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:27.986284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:27.986572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:27.986901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:27.987559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:27.995240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:27.999860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 856/1696] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:28.002535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.029779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df88391-911d-4144-b580-4ae83dc39da6 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.053927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 841/1697] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:27 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:28.059516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.071649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.072288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.075118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.075396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.076181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:28.076493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.078137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.115028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 857/1698] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:28.116240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.141225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0c4d9-a251-4da8-8a13-de7b55e2d807 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.278136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.278822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.279211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.279488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.280131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.287513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.333377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d59f9-30a9-4bf6-ac4f-0f1fc8cd3d5f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.351639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.352412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.353912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.355393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.357859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.363650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.393500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 858/1699] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:28 2022] POST /identity/v3/projects => generated 385 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:28.398663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.417369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 842/1700] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:28 2022] POST /identity/v3/users => generated 283 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:28.421334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.494570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a14d33-5cc3-4093-89dd-b591033f0da1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.535008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd5de3-8b71-46a3-8ff8-7f3027a7146c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.541441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.542175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.542525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.542893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.543613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.548019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.582191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 859/1701] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:28 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:28.587078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.618255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adf4f7-d29c-4b05-b914-c6b7001c554e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.643359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.644684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.648235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.648235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.648235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.662649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.663444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.663826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.664198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.664981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.668908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.670971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.704939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 843/1702] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:28.706358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.724902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:28.728514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 860/1703] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:28.733960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ecea6-bb9d-416d-aadf-93dccd53b2e4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.734413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.745598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.746248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.746643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.747038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.748265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.749991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.785642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab489d8b-07ce-4596-96e4-e4b601bad8ce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.800389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.802438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.802966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.803770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 844/1704] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:28.804607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.807875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.808042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.814135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.833177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39577c7-8962-40c2-ac83-47039894e6e7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.856080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.856759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.858738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 861/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:28.861228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.861356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:28.861518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] PATH_INFO: `/v3/projects/19f34fa4f5e64eb98e56722ff82dd2db/users/00401a8e3762479aa43e92ebcea45ad9/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.903718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa1892e-f42e-444b-b439-52a712075342 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:28.911346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:28.912626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.914132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.914804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] PATH_INFO: `/v3/projects/bc0c6b7d2ace491693a7c7556adc7de5/users/78f003162ae044c18eef9a053b086973/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.919476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:28.943647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f34fa4f5e64eb98e56722ff82dd2db, user_id=00401a8e3762479aa43e92ebcea45ad9, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.947302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4059508-9ec6-4997-a66a-86079677aec1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:28.987234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0c6b7d2ace491693a7c7556adc7de5, user_id=78f003162ae044c18eef9a053b086973, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:28.989633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 845/1706] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:28 2022] PUT /identity/v3/projects/19f34fa4f5e64eb98e56722ff82dd2db/users/00401a8e3762479aa43e92ebcea45ad9/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:28.991013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:28.991302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:28.991577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:28.992806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964ab90-67c3-4345-9a06-3e69cce427d3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:29.027165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 862/1707] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:28 2022] PUT /identity/v3/projects/bc0c6b7d2ace491693a7c7556adc7de5/users/78f003162ae044c18eef9a053b086973/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:29.039193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69868-c9bf-4434-924d-3cd042baa193 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.039193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69868-c9bf-4434-924d-3cd042baa193 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.039193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69868-c9bf-4434-924d-3cd042baa193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.101848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ecc69868-c9bf-4434-924d-3cd042baa193 None None] MFA Rules not processed for user `00401a8e3762479aa43e92ebcea45ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:29.104732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ecc69868-c9bf-4434-924d-3cd042baa193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:29.137372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:29.139181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:29.254666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 863/1708] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:29.261027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.261313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.261589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.302976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 846/1709] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:24:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:29.304129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:29.369856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] MFA Rules not processed for user `78f003162ae044c18eef9a053b086973`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:29.372117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:29.520505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9f45a-3c09-48d2-a8b0-d634fab1f94f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:29.539912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:29.540545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.540829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.541108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.541757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:29.543757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:29.550898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 864/1710] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:29.555466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:29.601125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b251bf-bec5-459c-9998-1dd86fb0ed03 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:29.616738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:29.617606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.617950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.618300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.619201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:29.623525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:29.636497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 847/1711] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/projects => generated 361 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:29.643930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:29.651608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:29.666404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 865/1712] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:29.667521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:29.740717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:29.770658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:29.784716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a44249-4329-400b-bb66-0be0c1fb8cfe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:29.801106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:29.802197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.802490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.802817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.805106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:29.811006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:29.863574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 866/1713] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:29.868428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:29.909006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192a8c5-0513-426d-bb7e-7d3732842865 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:29.920615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cfef89-3b0b-46f7-acf2-6b7f63f1b717 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:29.953220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:29.955939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:29.956249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:29.956528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:29.957201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:29.959226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.023158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 867/1714] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:29 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:30.032122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.058479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af3de9-389c-40a4-9d02-ca436c28f9f8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.070225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.072099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.072924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.081437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.082815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.085222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.109516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 868/1715] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:30.110606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.124942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceb522-de3d-4fa8-b36e-edad86adfbc6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.130893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.132202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.132555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.132978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.133713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.135830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.147375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.148569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.148569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.148569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.177274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 869/1716] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:30.177274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.207547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e835801-430d-478c-9c35-e62294b12080 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.238383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.245666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.246054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.246479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] PATH_INFO: `/v3/projects/5c2369e09c3148e3828ebb60970d5575/users/0d0b63363e874d24b81dfc5dc87b1798/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.293330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.295061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.314023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2369e09c3148e3828ebb60970d5575, user_id=0d0b63363e874d24b81dfc5dc87b1798, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.316073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 848/1717] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:24:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:24:30.316865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.327824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.345867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54180bb-e070-419c-b60e-5266aadc20eb service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.351319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.351988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.352274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.352551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] PATH_INFO: `/v3/projects/fbe14140387841f7af12530d84f816a5/users/ac6e66c2e7bd4adb97694559f998d445/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.377862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 870/1718] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:30 2022] PUT /identity/v3/projects/5c2369e09c3148e3828ebb60970d5575/users/0d0b63363e874d24b81dfc5dc87b1798/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:30.378887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.399908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe14140387841f7af12530d84f816a5, user_id=ac6e66c2e7bd4adb97694559f998d445, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.409555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.474797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 849/1719] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:30 2022] PUT /identity/v3/projects/fbe14140387841f7af12530d84f816a5/users/ac6e66c2e7bd4adb97694559f998d445/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:30.478508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.577504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:30.666996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9768773-ecbd-48a6-a05a-43a43794f48b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.719066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0f2d8-4204-45a5-a59a-6d4757ca130c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.741318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.743102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.743384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.743653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.744377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.750530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.801668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 850/1720] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:24:30.803785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.832314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6e627-f163-42ae-ab67-ff64927bd29e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.852683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.853369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.853649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.853925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] PATH_INFO: `/v3/projects/fbe14140387841f7af12530d84f816a5/users/ac6e66c2e7bd4adb97694559f998d445/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.859421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe14140387841f7af12530d84f816a5, user_id=ac6e66c2e7bd4adb97694559f998d445, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.862122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf35b6-1c45-4d09-9c17-9ae0a1d270ea admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.866345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:30.868998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.875004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.878906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.879795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:30.882076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:30.926624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 871/1721] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 539 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:30.926872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 851/1722] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:30 2022] PUT /identity/v3/projects/fbe14140387841f7af12530d84f816a5/users/ac6e66c2e7bd4adb97694559f998d445/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:30.931336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:30.933321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:30.933610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:30.933883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:30.956896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba5357-b171-4511-bbc0-dc916cad4f3a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:30.977486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:31.002222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] MFA Rules not processed for user `ac6e66c2e7bd4adb97694559f998d445`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:31.003529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:31.167045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:31.168289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:31.168607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:31.169037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] PATH_INFO: `/v3/projects/5c2369e09c3148e3828ebb60970d5575/users/0d0b63363e874d24b81dfc5dc87b1798/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:31.174362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2369e09c3148e3828ebb60970d5575, user_id=0d0b63363e874d24b81dfc5dc87b1798, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:31.177443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddddef-7b8a-405f-9865-80c6eea94e90 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:31.208647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 872/1723] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:30 2022] PUT /identity/v3/projects/5c2369e09c3148e3828ebb60970d5575/users/0d0b63363e874d24b81dfc5dc87b1798/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:31.221160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:31.224785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:31.225161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:31.272452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 852/1724] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:31.281079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:31.282647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:31.319812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] MFA Rules not processed for user `0d0b63363e874d24b81dfc5dc87b1798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:31.321084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:31.467886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320f845-3db9-4159-95e9-f8bbda903a60 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:31.500569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 873/1725] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:31.500569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:31.501712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:31.571407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:31.572396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:31.572967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:31.573401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:31.583544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:31.676263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d0f9a-a6d4-45d2-af21-9251b5e70c88 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:31.688112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:31.688655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:31.689787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:31.690188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] PATH_INFO: `/v3/projects/6ae79386aea24d858d7e8de0a4ae27b3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:31.696027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae79386aea24d858d7e8de0a4ae27b3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:31.698524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:31.764457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:31.765945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:31.776963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:31.802377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 853/1726] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:31.813921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae79386aea24d858d7e8de0a4ae27b3', 'request_id': 'req-40efae1f-78a4-44ac-affc-f5802113661a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:31.816612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:31.823454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:31.854113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae79386aea24d858d7e8de0a4ae27b3', 'request_id': 'req-40efae1f-78a4-44ac-affc-f5802113661a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:31.870160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdae75-8fc9-41a2-97f1-f9c0281d442c service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:31.870749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae79386aea24d858d7e8de0a4ae27b3', 'request_id': 'req-40efae1f-78a4-44ac-affc-f5802113661a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:31.912340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] The token cache is being invalidate because project 6ae79386aea24d858d7e8de0a4ae27b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:31.912768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:31.914328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-40efae1f-78a4-44ac-affc-f5802113661a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:31.922870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 874/1727] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:31 2022] DELETE /identity/v3/projects/6ae79386aea24d858d7e8de0a4ae27b3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:31.927766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7070-bfe2-492e-96ab-b1dce295acc3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:31.928209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7070-bfe2-492e-96ab-b1dce295acc3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:31.931675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7070-bfe2-492e-96ab-b1dce295acc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.017117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-032d7070-bfe2-492e-96ab-b1dce295acc3 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:32.018915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-032d7070-bfe2-492e-96ab-b1dce295acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.062570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:32.063549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.064905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.065337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.073169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.150056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 875/1728] 10.222.0.27 () {58 vars in 1015 bytes} [Sun Sep 18 06:24:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:32.151566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.154121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.154610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.207647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:32.209015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.273134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:32.277136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:32.311106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 854/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:32.312088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:32.319001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.432130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 876/1730] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:32.434813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:32.447467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.468737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:32.502639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052216a-4181-4a37-a7a0-d878ea094f1e service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:32.509990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:32.510955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.514727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.515343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] PATH_INFO: `/v3/users/78f003162ae044c18eef9a053b086973` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.518960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78f003162ae044c18eef9a053b086973)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:32.521954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:32.531946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa451be-63cb-4405-990e-047a6e0c7e40 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:32.547628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:32.548696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.549054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.549298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.574327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:32.578862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe48767-60c9-47e5-99b9-dbe560f85e7d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:32.603005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 877/1731] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:32.604751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.605933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.606359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.630816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:32.632043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.655895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 878/1732] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:32.655895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:32.655895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.687187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c5c55-122e-4801-925f-caef50b4e8c2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:32.692497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:32.693133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:32.693401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:32.693671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:32.705107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.727045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78f003162ae044c18eef9a053b086973'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:32.744315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78f003162ae044c18eef9a053b086973'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:32.745041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78f003162ae044c18eef9a053b086973'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:32.745655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78f003162ae044c18eef9a053b086973'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:32.772414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78f003162ae044c18eef9a053b086973'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:32.813452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 855/1733] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:32 2022] DELETE /identity/v3/users/78f003162ae044c18eef9a053b086973 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:32.814643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:32.817520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:32.875112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:32.942359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:32.949519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b84f9-56da-466b-b48f-7b480e6b7dfe service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:32.998698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 879/1734] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:33.017700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.017700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.017700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.066778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ab819-a63f-4808-a4d8-2328e7f9b440 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.085911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:33.086727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.087448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:33.088732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.089180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.090437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.094038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:33.096974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.098804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.125988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 880/1735] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:33.148497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 856/1736] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:33.153318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.167764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9d9e3-3726-4564-90fb-d351f91407ef None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.181281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.182218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.182633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.183051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.183817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.189051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.275552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 881/1737] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:33 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:33.277431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.300276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96253cd-7219-41e0-b224-9a05482925d5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.383966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.386006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:33.566372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.567054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.569252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.569252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.569659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.582032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89838b4a-d276-4b13-ac72-7d4d2a392c2e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.582569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.588240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.589165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.589725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.590263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.608895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:33.611063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.613377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.635667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 882/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:33.641874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 857/1739] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:33 2022] POST /identity/v3/users => generated 290 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:33.647239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.664740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7b04f-40fe-4ce4-9b3d-5dcd3022c05a service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.692802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.693823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.694302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.694779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.695673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.697989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.722545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 883/1740] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:33.726085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.747185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce198-2bf1-4715-81c0-0ca6147d8714 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.754465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.755135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.755562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.755994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.757198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.761329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39a5eb-0ca1-4cd6-996a-50c4e696f8c4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.812652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 858/1741] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:33.816043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:33.832238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1d12c-0f8b-421c-80c8-d6113c6c77e4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:33.840374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:33.841954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.842461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.842968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] PATH_INFO: `/v3/projects/503377a608a5486782ddc04cec6c9a8e/users/8ab19e41955c43eebe374d5e75af9a7d/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:33.898109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=503377a608a5486782ddc04cec6c9a8e, user_id=8ab19e41955c43eebe374d5e75af9a7d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:33.911421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:33.947884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 884/1742] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:33 2022] PUT /identity/v3/projects/503377a608a5486782ddc04cec6c9a8e/users/8ab19e41955c43eebe374d5e75af9a7d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:33.959342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:33.960001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:33.960001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:34.024666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] MFA Rules not processed for user `8ab19e41955c43eebe374d5e75af9a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:34.025922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:34.212668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 859/1743] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:34.217257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:34.395750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8609d8d-268f-4aa9-bd8f-1ca92ffebebe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:34.404048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:34.406498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:34.407603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:34.408594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:34.409375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:34.412727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:34.517442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 885/1744] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:34 2022] POST /identity/v3/projects => generated 381 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:34.518725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:34.542073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb69b8d-b404-4786-a2a8-e7be602e4f6b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:34.751343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:34.754410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:34.755104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:34.755593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:34.757529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:34.762058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:34.857851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:34.859730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:34.871884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 860/1745] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:34 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:34.882090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:34.905053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49615e-bd98-4bd5-97b3-2f910e34ae9b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:34.931844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:34.932514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:34.932816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:34.933120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] PATH_INFO: `/v3/projects/bc0c6b7d2ace491693a7c7556adc7de5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:34.948320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986e371-fad3-496f-bae4-6984086ad450 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:34.949462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0c6b7d2ace491693a7c7556adc7de5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:34.959663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:34.969439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:34.970172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:34.970320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:34.987927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:34.989152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:34.995671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:35.007520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:35.078969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 861/1746] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:35.079891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:35.126196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a047d8-c0b7-48a0-9de6-c07b75376139 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:35.153420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:35.154129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:35.154421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:35.154712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] PATH_INFO: `/v3/projects/0dedeba83ac5417791b5575fc21d6191/users/61675e35e9af44349dea5934ebf93e06/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:35.214683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0c6b7d2ace491693a7c7556adc7de5', 'request_id': 'req-668f873c-1502-4a2c-9bea-32a68b91aa8e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.232961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dedeba83ac5417791b5575fc21d6191, user_id=61675e35e9af44349dea5934ebf93e06, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:35.235898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:35.243885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0c6b7d2ace491693a7c7556adc7de5', 'request_id': 'req-668f873c-1502-4a2c-9bea-32a68b91aa8e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.266546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0c6b7d2ace491693a7c7556adc7de5', 'request_id': 'req-668f873c-1502-4a2c-9bea-32a68b91aa8e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.291827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 862/1747] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:35 2022] PUT /identity/v3/projects/0dedeba83ac5417791b5575fc21d6191/users/61675e35e9af44349dea5934ebf93e06/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:35.297597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:35.309714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] The token cache is being invalidate because project bc0c6b7d2ace491693a7c7556adc7de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:35.309714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.313461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.338666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 886/1748] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:34 2022] DELETE /identity/v3/projects/bc0c6b7d2ace491693a7c7556adc7de5 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:35.342589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:35.345388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:35.393534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:35.518967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d6c5-414e-4af0-8244-4eafbb21fc23 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:35.530039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:35.532160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:35.532636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:35.534813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:35.535684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:35.542845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:35.572872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 863/1749] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:35.573938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:35.581007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:35.617571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f873c-1502-4a2c-9bea-32a68b91aa8e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:35.627097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7850bc2-fdd8-4db9-a6a0-795ed3c4f973 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:35.634878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:35.636073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:35.636484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:35.636891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] PATH_INFO: `/v3/projects/0dedeba83ac5417791b5575fc21d6191/users/61675e35e9af44349dea5934ebf93e06/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:35.639711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dedeba83ac5417791b5575fc21d6191, user_id=61675e35e9af44349dea5934ebf93e06, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:35.639893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:35.640565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:35.640849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:35.641126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] PATH_INFO: `/v3/users/ac6e66c2e7bd4adb97694559f998d445` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:35.643285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43688551-9dc3-45ab-bb73-32d5d02abf70 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:35.652864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac6e66c2e7bd4adb97694559f998d445)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:35.659222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:35.668431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 864/1750] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:35 2022] PUT /identity/v3/projects/0dedeba83ac5417791b5575fc21d6191/users/61675e35e9af44349dea5934ebf93e06/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:35.670781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:35.671471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:35.672051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:35.711864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] MFA Rules not processed for user `61675e35e9af44349dea5934ebf93e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:35.713663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:35.853154 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac6e66c2e7bd4adb97694559f998d445'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.869057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac6e66c2e7bd4adb97694559f998d445'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.870782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac6e66c2e7bd4adb97694559f998d445'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.871976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac6e66c2e7bd4adb97694559f998d445'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.921548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac6e66c2e7bd4adb97694559f998d445'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:35.950450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 865/1751] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:35.956726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:35.958862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 887/1752] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:35 2022] DELETE /identity/v3/users/ac6e66c2e7bd4adb97694559f998d445 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:35.963314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:36.061019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c26c1c-e164-4c40-bba9-37b93f51e6ca None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:36.182946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:36.185788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:36.188284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:36.189523 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:36.203402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:36.361628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:36.363384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a988b0-a4d7-467b-9fa3-df758a322a6e service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:36.383118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 866/1753] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:36.528360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:36.530276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:36.600478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4997aba-e02f-45bc-aad7-8e439bd5c32c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:36.610459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:36.611476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:36.611932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:36.612356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:36.641756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:36.661326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:36.836312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:36.843468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9cecc-6479-4cfb-ab11-7c87279f7541 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:36.897156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 888/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:38.030860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:38.031427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:38.032516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:38.035878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 867/1755] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:24:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:38.508369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:38.510048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:38.510566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:38.565782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:38.574501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:38.873842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 889/1756] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:24:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:38.884681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:38.887106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:38.982468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799bfb9-812b-44d2-9542-11297d7a0a46 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:39.004359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:39.005756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:39.006108 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:39.006390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:39.033634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:39.035400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d610-13a7-4ed3-94cc-2a4ffbb13f20 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:39.060565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 868/1757] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:39.566725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:39.568893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:39.654001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad036-a511-4e29-96a9-3d653b217bb0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:39.660036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:39.660820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:39.661236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:39.661511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] PATH_INFO: `/v3/projects/fbe14140387841f7af12530d84f816a5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:39.662852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe14140387841f7af12530d84f816a5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:39.664885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:39.723780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:39.727227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe14140387841f7af12530d84f816a5', 'request_id': 'req-3471a405-3a32-42a2-a60e-b4eff521684f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:39.734298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe14140387841f7af12530d84f816a5', 'request_id': 'req-3471a405-3a32-42a2-a60e-b4eff521684f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:39.742274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe14140387841f7af12530d84f816a5', 'request_id': 'req-3471a405-3a32-42a2-a60e-b4eff521684f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:39.753531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] The token cache is being invalidate because project fbe14140387841f7af12530d84f816a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:39.753894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:39.754525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3471a405-3a32-42a2-a60e-b4eff521684f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:39.756332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 890/1758] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:39 2022] DELETE /identity/v3/projects/fbe14140387841f7af12530d84f816a5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:39.761284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:39.762600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:39.763173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:39.788454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:39.789858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:39.879408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 869/1759] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:39.887554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:39.887980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:39.888340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:39.925758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:39.927153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:39.947769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 891/1760] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:39.949016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:39.961298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1288ac5-3d3d-4b8c-bfb5-635741711264 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:39.970817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:39.972577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:39.973024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:39.973457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:39.975086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:39.975830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:39.978407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:39.995705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 870/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:40.000485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.010210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f615a-2efd-4fad-b5da-4eb9a03a66ef None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.015435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.016198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.016575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.016909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.017667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.019872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.061272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 892/1762] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:39 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:40.066185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.078749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3bcba-bb81-4916-985e-d76017b78e17 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.157618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.158263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.158541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.158845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.159516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.161474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.197998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 871/1763] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:40 2022] POST /identity/v3/users => generated 282 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:40.199551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.210182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca70e8a-6e4e-44a3-b5e2-3acb8b2d7f73 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.214950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.215651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.215970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.216342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.217054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.218771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.229550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 893/1764] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:40.234464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.259493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:40.261816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c487daf-e968-46d4-aeef-a0e6e3835859 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.270695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.281920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.282299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.282752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.284429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.287620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1917c-36cc-4550-b70c-09191fcea76a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.334626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 872/1765] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:40.341473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.351550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ceef84-2265-4370-b47d-e554adf99110 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.357251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.357932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.358233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.358537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] PATH_INFO: `/v3/projects/242344b2aa9d445d8504956b1fc5655c/users/ed8c2fc2f2bb44f1b172019b0e0158e2/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.377384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=242344b2aa9d445d8504956b1fc5655c, user_id=ed8c2fc2f2bb44f1b172019b0e0158e2, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.380568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6e7a2-dcd6-46dd-af0d-1940db4e36ef admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.406721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 894/1766] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:40 2022] PUT /identity/v3/projects/242344b2aa9d445d8504956b1fc5655c/users/ed8c2fc2f2bb44f1b172019b0e0158e2/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:40.420058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41277722-781d-46b6-aae2-4160bec6889e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.420689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41277722-781d-46b6-aae2-4160bec6889e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.421058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41277722-781d-46b6-aae2-4160bec6889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.472982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41277722-781d-46b6-aae2-4160bec6889e None None] MFA Rules not processed for user `ed8c2fc2f2bb44f1b172019b0e0158e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:40.474505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41277722-781d-46b6-aae2-4160bec6889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:40.522460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34a272-0ae0-442c-b6dc-06f5a5cd9c74 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.522701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34a272-0ae0-442c-b6dc-06f5a5cd9c74 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.522933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34a272-0ae0-442c-b6dc-06f5a5cd9c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.628712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8b34a272-0ae0-442c-b6dc-06f5a5cd9c74 None None] MFA Rules not processed for user `93d23721dfd34020b659707b65eb2c33`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:40.630040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b34a272-0ae0-442c-b6dc-06f5a5cd9c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:40.656581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 873/1767] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:40.663136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41277722-781d-46b6-aae2-4160bec6889e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.667976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41277722-781d-46b6-aae2-4160bec6889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:40.865633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 895/1768] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:40.876933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.877227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.877527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.966893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] MFA Rules not processed for user `93d23721dfd34020b659707b65eb2c33`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:40.968127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:40.969233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41277722-781d-46b6-aae2-4160bec6889e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:40.975407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:40.976143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:40.976313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:40.976581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:40.983671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 896/1769] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:40.988137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:40.989642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ad4cce-2129-4052-b03d-15650f606fe6 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:40.993318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:40.997410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:41.017889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 874/1770] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:41.117194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:41.158504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96307938-c28b-44f2-b121-51f92ed092a5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:41.175485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:41.176099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:41.176378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:41.176621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:41.194444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:41.196115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805212f-2b7f-4774-b88e-f839423da0f1 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:41.217042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 897/1771] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:42.026265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d8b0-37d6-4997-9465-48388cb6f158 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.027410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d8b0-37d6-4997-9465-48388cb6f158 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.027905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6d8b0-37d6-4997-9465-48388cb6f158 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.030071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 875/1772] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:24:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:42.049190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1f79-e1d8-4edc-9f3b-00d8035284c3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.049659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1f79-e1d8-4edc-9f3b-00d8035284c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.049659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1f79-e1d8-4edc-9f3b-00d8035284c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.092341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-740f1f79-e1d8-4edc-9f3b-00d8035284c3 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:42.093853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-740f1f79-e1d8-4edc-9f3b-00d8035284c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:42.124570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 898/1773] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:24:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:42.178971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff02e7-e83f-486f-9b17-13a3a6eae54f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.179246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff02e7-e83f-486f-9b17-13a3a6eae54f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.179765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff02e7-e83f-486f-9b17-13a3a6eae54f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.181799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 876/1774] 10.222.0.20 () {54 vars in 912 bytes} [Sun Sep 18 06:24:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:42.200207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.200755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.201177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.273481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:42.276083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:42.513410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 899/1775] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:24:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:42.554949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.555641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.555641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.588275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:42.590111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:42.632764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362cc36-3bab-4134-a595-4ba0de63ea38 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:42.655595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:42.668032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.668467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.668748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.714241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:42.715810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:42.738309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] MFA Rules not processed for user `d85160b01d824bc3a32b425fab98d2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:42.739835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:42.746929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 900/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:42.777033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:42.782236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:42.863535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797c9d5-904a-408b-9bcc-b81fe1c0f9de service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:42.890282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:42.890963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.891241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.891516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:42.902482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:42.904767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661deb74-4286-4a81-aee9-444beddbf76f service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:42.928909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 901/1777] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:42.930374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:42.930689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:42.930961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:43.025952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 877/1778] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:43.033277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] MFA Rules not processed for user `6c945a3e6a4946559aeadc63566d29f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:43.034928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:43.108999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:43.136888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a3a64-9532-4e24-8c31-fa1c39985447 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:43.165343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:43.165817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:43.173064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:43.173564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:43.175841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 902/1779] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:43.195638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:43.197592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65578e-5ee2-4bc9-99b9-ec2ad03c50e9 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:43.221193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 878/1780] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:43.402948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:43.419974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8beaa3-ab8a-42f3-a1e5-321e7f718860 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:43.426801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:43.427356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:43.427634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:43.427905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:43.438543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:43.440629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f8131-5668-4b7d-af78-317119b2fb43 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:43.455835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 903/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:46.766486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:46.767370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:46.767815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:46.780453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:46.806813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] MFA Rules not processed for user `d85160b01d824bc3a32b425fab98d2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:46.808302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:46.816955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 879/1782] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:46.850847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:46.851711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:46.852177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:46.891063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] MFA Rules not processed for user `6c945a3e6a4946559aeadc63566d29f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:46.892269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:46.900255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 904/1783] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:53.698275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:53.700264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:53.774915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ca1d-e565-4903-9808-48aa4ca32a65 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:53.775812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:53.777352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:53.780330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:53.780966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:53.781246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:53.781517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] PATH_INFO: `/v3/users/00401a8e3762479aa43e92ebcea45ad9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:53.782708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00401a8e3762479aa43e92ebcea45ad9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:53.784794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:53.794814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25a28f-84ca-40ea-a5e1-a35f77474161 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:53.800873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:53.801816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:53.802099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:53.802370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] PATH_INFO: `/v3/users/8ab19e41955c43eebe374d5e75af9a7d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:53.803995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab19e41955c43eebe374d5e75af9a7d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:53.806800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:53.832785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:53.852040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00401a8e3762479aa43e92ebcea45ad9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.864259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00401a8e3762479aa43e92ebcea45ad9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.866555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00401a8e3762479aa43e92ebcea45ad9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.867417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00401a8e3762479aa43e92ebcea45ad9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.871228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab19e41955c43eebe374d5e75af9a7d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.880519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b05d1a24-5e54-4d74-bcd2-7e332b1de0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00401a8e3762479aa43e92ebcea45ad9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.883787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab19e41955c43eebe374d5e75af9a7d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.885412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab19e41955c43eebe374d5e75af9a7d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.886344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab19e41955c43eebe374d5e75af9a7d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.901567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5bd2c1e-43bd-483c-b56c-ae129ae164b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab19e41955c43eebe374d5e75af9a7d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:53.902054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 880/1784] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:53 2022] DELETE /identity/v3/users/00401a8e3762479aa43e92ebcea45ad9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:53.907433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:53.907899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:53.908184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:53.915125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 905/1785] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:53 2022] DELETE /identity/v3/users/8ab19e41955c43eebe374d5e75af9a7d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:53.920006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:53.920282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:53.920558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:53.928564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:53.929869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:53.940447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:53.941648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:54.000301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 881/1786] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:54.010929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:54.012391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:54.014956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 906/1787] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:54.035262 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:54.037629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:54.039806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:54.101712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990092ae-5806-41df-bd4c-e2bed46f0742 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:54.107578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:54.108333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:54.109767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:54.110096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:54.143083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:54.145447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:54.169290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 882/1788] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:54.194916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c3d14-c91e-4938-b7be-a22fc3c55355 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:54.199982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:54.201751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:54.202217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:54.202909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:54.233941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:54.236407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:54.251081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 907/1789] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:24:55.293964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:55.295970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:55.317106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ed823-3fe1-4d3e-8c05-e008ca7dbbf2 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:55.335343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:55.336544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:55.337132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:55.337606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] PATH_INFO: `/v3/projects/19f34fa4f5e64eb98e56722ff82dd2db` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:55.338982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f34fa4f5e64eb98e56722ff82dd2db)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:55.341297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:55.447166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f34fa4f5e64eb98e56722ff82dd2db', 'request_id': 'req-914bb180-aee1-4239-b373-87e92b021063'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.466909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f34fa4f5e64eb98e56722ff82dd2db', 'request_id': 'req-914bb180-aee1-4239-b373-87e92b021063'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.478466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f34fa4f5e64eb98e56722ff82dd2db', 'request_id': 'req-914bb180-aee1-4239-b373-87e92b021063'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.499604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] The token cache is being invalidate because project 19f34fa4f5e64eb98e56722ff82dd2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:55.499928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.501234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.504633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 883/1790] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:55 2022] DELETE /identity/v3/projects/19f34fa4f5e64eb98e56722ff82dd2db => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:55.509551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:55.525574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc93cd-b79b-4bff-ae26-31568237b58c service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:55.594648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:55.645736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:55.646489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:55.646866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:55.647212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] PATH_INFO: `/v3/users/0d0b63363e874d24b81dfc5dc87b1798` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:55.649772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0b63363e874d24b81dfc5dc87b1798)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:55.651837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:55.763512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0b63363e874d24b81dfc5dc87b1798'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.774901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0b63363e874d24b81dfc5dc87b1798'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.775839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0b63363e874d24b81dfc5dc87b1798'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.776666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0b63363e874d24b81dfc5dc87b1798'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.839471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0b63363e874d24b81dfc5dc87b1798'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:55.873755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 908/1791] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:55 2022] DELETE /identity/v3/users/0d0b63363e874d24b81dfc5dc87b1798 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:56.013385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:56.015217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:56.035418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bb180-aee1-4239-b373-87e92b021063 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:56.192919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:56.194946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:56.195566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:56.196242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] PATH_INFO: `/v3/projects/503377a608a5486782ddc04cec6c9a8e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:56.197615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=503377a608a5486782ddc04cec6c9a8e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:56.201197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:56.304944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:56.312222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503377a608a5486782ddc04cec6c9a8e', 'request_id': 'req-010f84da-07da-4a16-9e70-61e95379bd33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.326808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503377a608a5486782ddc04cec6c9a8e', 'request_id': 'req-010f84da-07da-4a16-9e70-61e95379bd33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.344208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503377a608a5486782ddc04cec6c9a8e', 'request_id': 'req-010f84da-07da-4a16-9e70-61e95379bd33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.372735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] The token cache is being invalidate because project 503377a608a5486782ddc04cec6c9a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:56.373535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.375480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.379140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 884/1792] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:56 2022] DELETE /identity/v3/projects/503377a608a5486782ddc04cec6c9a8e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:56.383640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:56.385188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:56.416283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1187-ef53-4927-a254-e9d790e89f05 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:56.559389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:56.561301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:56.561670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:56.562052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] PATH_INFO: `/v3/users/61675e35e9af44349dea5934ebf93e06` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:56.564941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] RBAC: Authorizing `identity:delete_user(user_id=61675e35e9af44349dea5934ebf93e06)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:56.568525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:56.649227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61675e35e9af44349dea5934ebf93e06'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.663718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61675e35e9af44349dea5934ebf93e06'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.664882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61675e35e9af44349dea5934ebf93e06'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.667694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61675e35e9af44349dea5934ebf93e06'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.676495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:56.688875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6f4c5897-a615-4322-8b00-23977d38534a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61675e35e9af44349dea5934ebf93e06'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:56.707793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 909/1793] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:24:56 2022] DELETE /identity/v3/users/61675e35e9af44349dea5934ebf93e06 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:56.852684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:56.854447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:56.887532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50473-fa15-41e4-b604-3cf11a48c6a4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:56.888062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50473-fa15-41e4-b604-3cf11a48c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:56.888402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d50473-fa15-41e4-b604-3cf11a48c6a4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:56.891572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 910/1794] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:24:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:24:56.894639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f84da-07da-4a16-9e70-61e95379bd33 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:56.908624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c550-30c3-45cb-99b2-202f073a9ca7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:56.909429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c550-30c3-45cb-99b2-202f073a9ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:56.909798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c550-30c3-45cb-99b2-202f073a9ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:56.988870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-16f2c550-30c3-45cb-99b2-202f073a9ca7 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:56.993406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16f2c550-30c3-45cb-99b2-202f073a9ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:57.133184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:57.134351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.135456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.136715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] PATH_INFO: `/v3/projects/5c2369e09c3148e3828ebb60970d5575` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.148390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2369e09c3148e3828ebb60970d5575)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:57.156326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:57.217356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 911/1795] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:24:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:57.300586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2369e09c3148e3828ebb60970d5575', 'request_id': 'req-7150e03d-0e44-4284-a4a6-8baf4576e050'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:57.318013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2369e09c3148e3828ebb60970d5575', 'request_id': 'req-7150e03d-0e44-4284-a4a6-8baf4576e050'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:57.336421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2369e09c3148e3828ebb60970d5575', 'request_id': 'req-7150e03d-0e44-4284-a4a6-8baf4576e050'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:57.365202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] The token cache is being invalidate because project 5c2369e09c3148e3828ebb60970d5575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:57.366728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:57.369015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:57.378161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 885/1796] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:56 2022] DELETE /identity/v3/projects/5c2369e09c3148e3828ebb60970d5575 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:57.380361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0872ad-887a-4621-ae35-f57f22e2fd12 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.380603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0872ad-887a-4621-ae35-f57f22e2fd12 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.380811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0872ad-887a-4621-ae35-f57f22e2fd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.439762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9f0872ad-887a-4621-ae35-f57f22e2fd12 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:57.447605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9f0872ad-887a-4621-ae35-f57f22e2fd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:57.568293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:57.569697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:57.637642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 912/1797] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:57.646613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.648572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.649049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.685791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:57.688885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:57.715062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:57.717409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 913/1798] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:57.722003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:57.737359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b8a93-ef56-4604-ba8d-a4cb68d02bd0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:57.744215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:57.744908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.745308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.745630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.749625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150e03d-0e44-4284-a4a6-8baf4576e050 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:57.751016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:57.751492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:57.754483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:57.757240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:57.758164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.759017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.759548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.762265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:57.777342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:57.785392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:57.789559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 914/1799] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:24:57.796318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:57.799524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 886/1800] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:24:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:24:57.809638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd321477-8f46-479c-b133-6eedc45f7cbe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:57.818273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:57.819149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:57.819431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:57.819882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:57.820630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:57.823547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:57.872970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 915/1801] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:57 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:57.873550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:57.910737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625962-c282-446e-8c57-e65a26d6b693 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:57.955723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:57.957646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:58.040825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:58.085535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.086300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.086711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.087709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.088485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.091621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.141222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 887/1802] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:57 2022] POST /identity/v3/users => generated 292 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:58.144333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.150168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb4f93-d422-41b0-8ceb-7b6d1a4f1fa1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.161861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4dbec-3f4c-4a25-9feb-d3940f7df369 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.163911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.164661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.165109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.165453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] PATH_INFO: `/v3/projects/0dedeba83ac5417791b5575fc21d6191` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.167076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dedeba83ac5417791b5575fc21d6191)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.169622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.180777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.181580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.181969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.182851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.184690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.187016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.207140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 888/1803] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:24:58.208503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.221028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c278ae-7ce3-4ef2-abdd-719fe8361341 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.227469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.228255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.228700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.228980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.230391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.234227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.255909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 889/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:58.257221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.272437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3fa2-797f-46b3-98d2-faaaf11f8c19 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.313618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dedeba83ac5417791b5575fc21d6191', 'request_id': 'req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:58.328944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dedeba83ac5417791b5575fc21d6191', 'request_id': 'req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:58.349728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dedeba83ac5417791b5575fc21d6191', 'request_id': 'req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:58.369524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] The token cache is being invalidate because project 0dedeba83ac5417791b5575fc21d6191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:24:58.369822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:58.371555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-082efa4d-b20e-4a7d-b728-1413d9bd2b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:24:58.374185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 916/1805] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:24:57 2022] DELETE /identity/v3/projects/0dedeba83ac5417791b5575fc21d6191 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:58.379309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018936-4d2b-48fc-9780-2f87ebb628bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.379606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018936-4d2b-48fc-9780-2f87ebb628bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.379880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb018936-4d2b-48fc-9780-2f87ebb628bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.399846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.400530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.400810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.401084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] PATH_INFO: `/v3/projects/56388a31e2074c87906b0bb4526e4f44/users/bf7f2ad3a07f462ab52abeba4f37f9f9/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.427563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56388a31e2074c87906b0bb4526e4f44, user_id=bf7f2ad3a07f462ab52abeba4f37f9f9, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.430331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355d5d2-b8b4-4b30-90b9-86730978d66a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.435934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-eb018936-4d2b-48fc-9780-2f87ebb628bc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:58.437065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-eb018936-4d2b-48fc-9780-2f87ebb628bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:58.450628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 917/1806] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:58.455264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 890/1807] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:58 2022] PUT /identity/v3/projects/56388a31e2074c87906b0bb4526e4f44/users/bf7f2ad3a07f462ab52abeba4f37f9f9/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:58.457814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.458104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.458382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.477158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.477723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.478240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.517802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:58.519773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:58.534852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] MFA Rules not processed for user `bf7f2ad3a07f462ab52abeba4f37f9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:58.538080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:58.604588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 918/1808] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:24:58.609695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.622161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b23b4-7a66-4656-bb99-b8ffc425b6fc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.627932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.628593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.629057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.629368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.630123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:24:58.630430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.633071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.641332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:58.661909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 919/1809] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:24:58.663447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.676267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0205d-11cf-4b70-bffd-c39c207c4ad8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.681416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.682376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.683204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.683493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.685226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.687568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.707706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 891/1810] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:58.725555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.729450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:58.747802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0177b41-9de9-4b2a-9739-f663aca878ea None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.753732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.755298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.756361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.756598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.757582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.759997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:58.768223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 920/1811] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:58.770285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.814818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:58.820717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 892/1812] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:58.825271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:58.915892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b1b397-2a67-411e-81c3-029bf3c56fcb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.968391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e614-066a-4560-af3f-bbd4e500399c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:58.986362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:58.987085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:58.988214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:58.988214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:58.988407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:58.992460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.044867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 893/1813] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:59.046745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.059894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00707ad5-b9e0-4381-b2f8-c9cc44493833 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.066507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.068102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.068616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.068948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.069794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.071268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.071931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.072317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.072742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.075052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.076040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.079029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.093089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 894/1814] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:59.095170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.115771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8099d0-5954-48c2-99fb-e050699673e7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.129643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 921/1815] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:58 2022] POST /identity/v3/users => generated 285 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:59.131911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.133957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.136757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.138408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.138408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.140642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.143736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.153752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a516e-6dd7-44f8-9b6f-70de808b8bbb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.160541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.161169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.161452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.161729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.162432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.164503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.169933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 895/1816] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:24:59.174567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.178159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 922/1817] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:59.182560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.189212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afbd03-ef7d-4fa7-bb79-e7728233a143 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.194444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171e16-1030-4ddb-9ab1-a895f14da1ee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.196708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.197683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.198052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.198631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] PATH_INFO: `/v3/projects/9fb59ae6d8a24c119a6c7ffde922ed83/users/2304b89f4d4b402bb6f684fba3034955/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.202370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.202370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.202370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.202370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.202896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.204529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.218615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 923/1818] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:24:59.218757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb59ae6d8a24c119a6c7ffde922ed83, user_id=2304b89f4d4b402bb6f684fba3034955, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.220348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03eb42-8dbf-4af6-96aa-dc8d73a7ce88 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.221772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.240198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6c878-edfb-4d79-94ce-c3ac265a3abc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.241665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 896/1819] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:59 2022] PUT /identity/v3/projects/9fb59ae6d8a24c119a6c7ffde922ed83/users/2304b89f4d4b402bb6f684fba3034955/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:24:59.252701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.253780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.254267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.302228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] MFA Rules not processed for user `2304b89f4d4b402bb6f684fba3034955`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:59.304134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:59.344275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.345005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.345449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.345601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] PATH_INFO: `/v3/projects/0930f427ce894b1f9a3f95c1c5b675e6/users/9a8d297b10784bf9853bc1440a767c4d/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.356545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:59.365706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0930f427ce894b1f9a3f95c1c5b675e6, user_id=9a8d297b10784bf9853bc1440a767c4d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.368471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718ec16-9448-4b21-9f38-f93066e49132 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.384812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 924/1820] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:59 2022] PUT /identity/v3/projects/0930f427ce894b1f9a3f95c1c5b675e6/users/9a8d297b10784bf9853bc1440a767c4d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:59.394092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 897/1821] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:59.395796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.395933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.396160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.396444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.416306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971be869-21c6-4f66-9ef6-4a46887c31c7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.450320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] MFA Rules not processed for user `9a8d297b10784bf9853bc1440a767c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:24:59.452569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:24:59.460198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:59.505643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.507008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.507537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.509203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.514725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.522361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.567419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 898/1822] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:59.571454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.595902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 925/1823] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:24:59.601574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.615540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a9792-df02-45c3-95ce-c536e0f30840 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.678194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9c515-49d6-43c3-a8ba-c481ac5abb62 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.682916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.683845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.684130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.684401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.685050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.685155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.685829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.686176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.686517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.687095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.687398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.689518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.726902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 926/1824] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:24:59.730951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.731572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 899/1825] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:59.735270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.746606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7595757-4162-459f-8ae3-ebd4af884926 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.821381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa05d2-d8a2-444c-819c-b57ee49f5e23 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.827725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.828472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.829818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.829818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.830378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.832756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.836377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:59.864058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 927/1826] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:24:59 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:24:59.864058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.865426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.866370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.866997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.867634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.868553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.870634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.875689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4a17-93c0-417f-85f2-82174fe08814 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.880632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.881254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.881534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.881809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.882457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.884382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.889206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 900/1827] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:59.892919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.901348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 928/1828] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:24:59.907693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.908543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51507c-cb0c-41bd-8d45-d41ee0008b21 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.914628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.915529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.916011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.916286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] PATH_INFO: `/v3/projects/0d3845d063e04633afc306ecb3637a31/users/0e44217fd3db4bd2aacea3ff93af0a9b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.934115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3845d063e04633afc306ecb3637a31, user_id=0e44217fd3db4bd2aacea3ff93af0a9b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.937094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:24:59.938194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d037286-63e0-4de5-9819-87f08e6047dd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:24:59.961378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:24:59.962524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:24:59.962524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:24:59.962923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] PATH_INFO: `/v3/projects/226e037874544304b7e53f4105799afa/users/f3c998e16b684f8aa577ec4a66887d8b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:24:59.967518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:24:59.971364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 901/1829] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:59 2022] PUT /identity/v3/projects/0d3845d063e04633afc306ecb3637a31/users/0e44217fd3db4bd2aacea3ff93af0a9b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:24:59.975319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:24:59.985127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=226e037874544304b7e53f4105799afa, user_id=f3c998e16b684f8aa577ec4a66887d8b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:24:59.988237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.008680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 929/1830] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:24:59 2022] PUT /identity/v3/projects/226e037874544304b7e53f4105799afa/users/f3c998e16b684f8aa577ec4a66887d8b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:00.009764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:00.053643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6650b1-eab6-4921-b2d3-c186016b4dd6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.090935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bfb6-a6bf-442e-a71f-3848eb681cbe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.096800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.097529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.097853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.098122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.099232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.101293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.116394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 930/1831] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:00.117402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:00.123199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.123965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.124243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.125000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.125205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.127014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.128471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffd91e-1c48-4130-8eed-c07c56d65260 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.134121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.134929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.135176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.135388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] PATH_INFO: `/v3/projects/226e037874544304b7e53f4105799afa/users/f3c998e16b684f8aa577ec4a66887d8b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.138041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226e037874544304b7e53f4105799afa, user_id=f3c998e16b684f8aa577ec4a66887d8b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.139408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 902/1832] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:00.140926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa276f-eb5d-461d-b5ba-bfccbcb58732 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.143323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:00.152449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 931/1833] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:00 2022] PUT /identity/v3/projects/226e037874544304b7e53f4105799afa/users/f3c998e16b684f8aa577ec4a66887d8b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:00.153216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c9cc6-acc4-4e9d-b0aa-18e583e164df admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.162637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.162948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.163330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.199370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] MFA Rules not processed for user `f3c998e16b684f8aa577ec4a66887d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:00.200644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:00.218083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.218875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.219182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.219497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] PATH_INFO: `/v3/projects/0d3845d063e04633afc306ecb3637a31/users/0e44217fd3db4bd2aacea3ff93af0a9b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.221637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3845d063e04633afc306ecb3637a31, user_id=0e44217fd3db4bd2aacea3ff93af0a9b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.224950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6651aa-e23e-4acc-b65b-0e05f4a06306 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.239714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 903/1834] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:00 2022] PUT /identity/v3/projects/0d3845d063e04633afc306ecb3637a31/users/0e44217fd3db4bd2aacea3ff93af0a9b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:00.251316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.251735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.252106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.263623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:00.286527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 932/1835] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:00.290801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] MFA Rules not processed for user `0e44217fd3db4bd2aacea3ff93af0a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:00.292994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:00.299807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:00.303648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:00.368871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 904/1836] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:00.380593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:00.382019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:00.389806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1a52e-08a0-4163-9985-ae76c164b720 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.394852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:00.444080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.445149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.445495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.445831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.446023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722a302-aa0c-4aba-bd84-b8a66f48bc1a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:00.451463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:00.452592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:00.453057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:00.454143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:00.471842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.473355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.484317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 905/1837] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:00.524051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:00.525854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:00.538030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 933/1838] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:01.420843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:01.431368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8412c7-c445-4621-89a8-bb512120cc0a service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:01.438794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:01.438794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:01.438794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:01.438794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:01.555283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:01.559341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e8bee-15d8-4659-8862-76b1ac7d3f28 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:01.572158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 906/1839] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:02.527078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:02.599791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311be53-6472-4006-b2ce-e447af36d5fa service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:02.604376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:02.605009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:02.605726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:02.605726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] PATH_INFO: `/v3/users/9a8d297b10784bf9853bc1440a767c4d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:02.606785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8d297b10784bf9853bc1440a767c4d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:02.608732 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:02.660191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8d297b10784bf9853bc1440a767c4d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:02.667138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8d297b10784bf9853bc1440a767c4d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:02.668378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8d297b10784bf9853bc1440a767c4d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:02.669025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8d297b10784bf9853bc1440a767c4d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:02.674815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:02.685306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8d297b10784bf9853bc1440a767c4d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:02.697503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 934/1840] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:02 2022] DELETE /identity/v3/users/9a8d297b10784bf9853bc1440a767c4d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:02.702035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:02.702524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:02.702647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:02.720983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:02.722199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:02.746681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:02.780603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 907/1841] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:02.790381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:02.791913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:02.848265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fd7ce-d377-4227-87fb-6113bfb3fbde admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:02.851865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:02.852472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:02.852751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:02.853021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:02.862432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:02.863942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:02.874018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 935/1842] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:03.141561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:03.143084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:03.158781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb7b4-395a-4b19-b65f-41bda6bf272e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:03.161032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:03.161032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:03.161032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:03.161385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] PATH_INFO: `/v3/projects/0930f427ce894b1f9a3f95c1c5b675e6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:03.163227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0930f427ce894b1f9a3f95c1c5b675e6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:03.165452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:03.226187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0930f427ce894b1f9a3f95c1c5b675e6', 'request_id': 'req-6e8af74f-991f-4bdf-925d-3a2c88749985'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.240339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0930f427ce894b1f9a3f95c1c5b675e6', 'request_id': 'req-6e8af74f-991f-4bdf-925d-3a2c88749985'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.248565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0930f427ce894b1f9a3f95c1c5b675e6', 'request_id': 'req-6e8af74f-991f-4bdf-925d-3a2c88749985'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.259670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] The token cache is being invalidate because project 0930f427ce894b1f9a3f95c1c5b675e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:03.259965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.260618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.262827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 908/1843] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:03 2022] DELETE /identity/v3/projects/0930f427ce894b1f9a3f95c1c5b675e6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:03.266789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:03.276606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365227e3-cc8e-4794-997d-57f2494d18ff service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:03.333942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:03.334601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:03.334886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:03.335153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] PATH_INFO: `/v3/users/f3c998e16b684f8aa577ec4a66887d8b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:03.336290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c998e16b684f8aa577ec4a66887d8b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:03.338207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:03.392287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c998e16b684f8aa577ec4a66887d8b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.399379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c998e16b684f8aa577ec4a66887d8b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.399988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c998e16b684f8aa577ec4a66887d8b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.400583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c998e16b684f8aa577ec4a66887d8b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.412136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fe1d4b1-001d-4f87-8583-10a63aa2a3d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c998e16b684f8aa577ec4a66887d8b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:03.420877 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:03.425906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 936/1844] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:03 2022] DELETE /identity/v3/users/f3c998e16b684f8aa577ec4a66887d8b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:04.233265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:04.235633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:04.265100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8af74f-991f-4bdf-925d-3a2c88749985 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:04.368632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:04.370214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:04.370727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:04.371267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] PATH_INFO: `/v3/projects/226e037874544304b7e53f4105799afa` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:04.373753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] RBAC: Authorizing `identity:delete_project(project_id=226e037874544304b7e53f4105799afa)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:04.391037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:04.525964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226e037874544304b7e53f4105799afa', 'request_id': 'req-7c15e080-3677-49dd-9b27-45b28058772f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:04.544052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226e037874544304b7e53f4105799afa', 'request_id': 'req-7c15e080-3677-49dd-9b27-45b28058772f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:04.546297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:04.555332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226e037874544304b7e53f4105799afa', 'request_id': 'req-7c15e080-3677-49dd-9b27-45b28058772f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:04.597100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] The token cache is being invalidate because project 226e037874544304b7e53f4105799afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:04.597393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:04.598185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c15e080-3677-49dd-9b27-45b28058772f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:04.599967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 909/1845] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:04 2022] DELETE /identity/v3/projects/226e037874544304b7e53f4105799afa => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:04.605697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:04.606163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:04.606369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:04.652161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:04.655765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:04.881396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 937/1846] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:04.891851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:04.892450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:04.892954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:04.993550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:04.995044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:05.030174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 910/1847] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:05.031598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.049994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378434b-8e1f-433e-a539-ab3d846fd723 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.093417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.094066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.094382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.094771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.095542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:05.095756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.097441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.145934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 938/1848] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:05.150221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.163063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fcf7-4431-4ccd-a912-17d87388b4d7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.186853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.188421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.190115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.191380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.196346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.203384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.283063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 911/1849] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:05 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:05.284342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.313228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2b047-c687-4b48-b300-d7da6779800e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.474956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.475814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.476162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.476440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.477110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.479227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.510655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 939/1850] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:05 2022] POST /identity/v3/users => generated 301 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:05.515293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.526084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653cc15-ea12-40e3-854a-742d7d4fd504 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.531170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.531864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.532208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.532590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.533425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.535184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.550150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 912/1851] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:05.563853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.582506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9e947-9b35-4cfd-9acf-85d6353b2278 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.587837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.588526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.588805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.589076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.589917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.591529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589703a7-e974-46ae-9fe6-3bf202819ad2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.602761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 940/1852] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:05.606931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.616860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1264-b9fa-4ac4-a190-51dfaf2f0409 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.622117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.622792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.623070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.623338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] PATH_INFO: `/v3/projects/a5a2ceb109e54ff099b467ff034af784/users/b04c6e37ebb7421cba3ff882275118b4/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.639906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a2ceb109e54ff099b467ff034af784, user_id=b04c6e37ebb7421cba3ff882275118b4, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.642836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.656983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 913/1853] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:05 2022] PUT /identity/v3/projects/a5a2ceb109e54ff099b467ff034af784/users/b04c6e37ebb7421cba3ff882275118b4/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:05.667216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.667530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.668453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.696812 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] MFA Rules not processed for user `b04c6e37ebb7421cba3ff882275118b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:05.698026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:05.776773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:05.793180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 941/1854] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:05.798628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.906150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ee130-dc7e-41fa-9d34-15c4d4c67b83 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:05.910989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:05.911685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:05.912028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:05.912390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:05.913098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:05.915210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:05.950620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 914/1855] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:05 2022] POST /identity/v3/projects => generated 401 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:05.951786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:05.962381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d2b5-5f46-40d9-9f2f-0c2a5f9059ff None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.018695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.019281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.019556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.019828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.020477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.022402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.049270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 942/1856] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:05 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:06.053572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.064998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c544-6b88-43bb-a3ef-8becd91c3cb5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.070826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.071454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.071815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.072486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.073703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.075461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.093350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 915/1857] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:06.098339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.110815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3c180-eba9-43e3-8b2e-a109e160a0fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.115643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.116372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.116711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.117045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] PATH_INFO: `/v3/projects/cf196a9b85764eb0a25d8329a1114fb5/users/fb68250f370d48eab22b602b98c8cdae/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.133801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf196a9b85764eb0a25d8329a1114fb5, user_id=fb68250f370d48eab22b602b98c8cdae, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.137304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.151338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 943/1858] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:06 2022] PUT /identity/v3/projects/cf196a9b85764eb0a25d8329a1114fb5/users/fb68250f370d48eab22b602b98c8cdae/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:06.156050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.166876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b70c2-dae5-402f-a1f9-a1d52a0106e3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.258655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:06.271436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.272377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.272753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.273119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.273852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.275692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d00cb7-5a5e-4d08-bec5-e4c6a955f485 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.286616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 916/1859] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:06.291671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.305434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c632f-a202-4936-be39-7ff63aae9d3f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.311973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.312865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.313269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.313642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] PATH_INFO: `/v3/projects/cf196a9b85764eb0a25d8329a1114fb5/users/fb68250f370d48eab22b602b98c8cdae/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.324293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf196a9b85764eb0a25d8329a1114fb5, user_id=fb68250f370d48eab22b602b98c8cdae, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.329074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.354444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 944/1860] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:06 2022] PUT /identity/v3/projects/cf196a9b85764eb0a25d8329a1114fb5/users/fb68250f370d48eab22b602b98c8cdae/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:06.356777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.357061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.357328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.424574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] MFA Rules not processed for user `fb68250f370d48eab22b602b98c8cdae`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:06.426082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:06.573519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.576577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:06.605511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 917/1861] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:06 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:06.661881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c56c41-e21d-416d-ab60-bdad488cbd18 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.665667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.666462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.666701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.667151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.669325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:06.685960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.687614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:06.703584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a83a18-14e5-494f-995d-921f0e9c10f3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.707572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.708188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.708513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.708780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:06.719680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.722400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.736167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 918/1862] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:06.752891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:06.754780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:06.767016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 945/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:06.933212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:06.944230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dcce-42d5-41bc-ad95-b12bf5de0ff8 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:06.948380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:06.949104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:06.949455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:06.949786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:07.028268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:07.029817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:07.039841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 919/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:06 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:09.333513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:09.336074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:09.356113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:09.500232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf3066-985b-4bda-99ba-1cfccbae5e25 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:09.505623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:09.506173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:09.506455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:09.506802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] PATH_INFO: `/v3/users/bf7f2ad3a07f462ab52abeba4f37f9f9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:09.508536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7f2ad3a07f462ab52abeba4f37f9f9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:09.512595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:09.534434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:09.537435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:09.581184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7f2ad3a07f462ab52abeba4f37f9f9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.592060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7f2ad3a07f462ab52abeba4f37f9f9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.593352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7f2ad3a07f462ab52abeba4f37f9f9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.595928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7f2ad3a07f462ab52abeba4f37f9f9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.607461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c7c4-a651-441b-90da-7fb43e23cf52 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:09.622634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:09.623334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:09.623616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:09.623893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] PATH_INFO: `/v3/users/1ad67acd84524ffc915c6105b0fe533d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:09.624950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c545308b-29a1-4a76-96e8-8f4e8f09c4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7f2ad3a07f462ab52abeba4f37f9f9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.627333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ad67acd84524ffc915c6105b0fe533d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:09.629274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:09.665716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 946/1865] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:09 2022] DELETE /identity/v3/users/bf7f2ad3a07f462ab52abeba4f37f9f9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:09.670707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:09.671141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:09.671548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:09.703318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:09.705396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:09.738960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ad67acd84524ffc915c6105b0fe533d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.764420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ad67acd84524ffc915c6105b0fe533d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.766149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ad67acd84524ffc915c6105b0fe533d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.767064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ad67acd84524ffc915c6105b0fe533d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.768824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:09.781895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a44b15a8-2576-4bf3-9215-3eedf35e797f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ad67acd84524ffc915c6105b0fe533d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:09.811297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 947/1866] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:09.823026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:09.827301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:09.841745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 920/1867] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:09 2022] DELETE /identity/v3/users/1ad67acd84524ffc915c6105b0fe533d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:09.841745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:09.841745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:09.841745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:09.871638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:09.873029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:09.881296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:09.939883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cac3-9e6e-41a2-9cc2-d280ca30c664 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:10.001311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 921/1868] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:10.011959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:10.032885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:10.033520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:10.033793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:10.034063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:10.065459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:10.067300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:10.112459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627dd0e-b2c0-4a9f-9ac6-b2ba241cc5c7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:10.118112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 948/1869] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:10.118893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:10.119594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:10.119950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:10.120328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:10.144451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:10.146020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:10.179732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 922/1870] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:10.725784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:10.726468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:10.728114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:10.729345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:10.778885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7fc9-1e54-46ba-8000-98f179c9308f service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:10.782865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a23901-ce02-4362-ba29-e8caee2b8eb5 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:10.805963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:10.806768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:10.806996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:10.807268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] PATH_INFO: `/v3/projects/d81b5577d70b4f55b0a48b7368814468` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:10.813118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:10.813841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:10.814167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:10.814429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] PATH_INFO: `/v3/projects/56388a31e2074c87906b0bb4526e4f44` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:10.815899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81b5577d70b4f55b0a48b7368814468)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:10.816689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] RBAC: Authorizing `identity:delete_project(project_id=56388a31e2074c87906b0bb4526e4f44)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:10.830701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:10.834147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:10.945647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56388a31e2074c87906b0bb4526e4f44', 'request_id': 'req-dfafaa34-db57-42dd-8b8b-086abfdc688d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.950321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81b5577d70b4f55b0a48b7368814468', 'request_id': 'req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.955932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56388a31e2074c87906b0bb4526e4f44', 'request_id': 'req-dfafaa34-db57-42dd-8b8b-086abfdc688d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.958410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81b5577d70b4f55b0a48b7368814468', 'request_id': 'req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.966096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56388a31e2074c87906b0bb4526e4f44', 'request_id': 'req-dfafaa34-db57-42dd-8b8b-086abfdc688d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.973446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81b5577d70b4f55b0a48b7368814468', 'request_id': 'req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.986410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:10.986977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] The token cache is being invalidate because project 56388a31e2074c87906b0bb4526e4f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:10.987651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.990371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.994481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] The token cache is being invalidate because project d81b5577d70b4f55b0a48b7368814468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:10.995038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:10.995881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 949/1871] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:10 2022] DELETE /identity/v3/projects/56388a31e2074c87906b0bb4526e4f44 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:10.997547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.000124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 923/1872] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:10 2022] DELETE /identity/v3/projects/d81b5577d70b4f55b0a48b7368814468 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:11.004046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:11.006025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:11.008078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:11.012210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:11.112588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9e45c-29b5-45f7-9b3d-11ffbe17a120 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:11.118527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:11.119435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:11.121326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:11.121326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] PATH_INFO: `/v3/users/20db2aecaee04f8d87a600aa76b3bf10` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:11.122117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20db2aecaee04f8d87a600aa76b3bf10)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:11.124288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:11.144845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafaa34-db57-42dd-8b8b-086abfdc688d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:11.154495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:11.155726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:11.156008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:11.156275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] PATH_INFO: `/v3/users/2304b89f4d4b402bb6f684fba3034955` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:11.158312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2304b89f4d4b402bb6f684fba3034955)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:11.160283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:11.165761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:11.203032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20db2aecaee04f8d87a600aa76b3bf10'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.217646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20db2aecaee04f8d87a600aa76b3bf10'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.218847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20db2aecaee04f8d87a600aa76b3bf10'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.219507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20db2aecaee04f8d87a600aa76b3bf10'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.224325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2304b89f4d4b402bb6f684fba3034955'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.227898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20db2aecaee04f8d87a600aa76b3bf10'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.234405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2304b89f4d4b402bb6f684fba3034955'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.235742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2304b89f4d4b402bb6f684fba3034955'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.236620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2304b89f4d4b402bb6f684fba3034955'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.249342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 924/1873] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:11 2022] DELETE /identity/v3/users/20db2aecaee04f8d87a600aa76b3bf10 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:11.253045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2304b89f4d4b402bb6f684fba3034955'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.268994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 950/1874] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:11 2022] DELETE /identity/v3/users/2304b89f4d4b402bb6f684fba3034955 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:11.700176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:11.701764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:11.754838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:11.756354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:11.857757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c9234f-081a-482d-aa0d-a37bc896ddc9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:11.858715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9d511-ef87-4c14-9e29-eb5f6cff4249 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:11.862713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:11.863354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:11.863525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:11.863651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:11.863927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] PATH_INFO: `/v3/projects/83a714b488774bc5a96530181a69e73a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:11.864091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:11.864370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:11.864636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] PATH_INFO: `/v3/projects/9fb59ae6d8a24c119a6c7ffde922ed83` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:11.865046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=83a714b488774bc5a96530181a69e73a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:11.865742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb59ae6d8a24c119a6c7ffde922ed83)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:11.867059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:11.868272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:11.890957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:11.916950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83a714b488774bc5a96530181a69e73a', 'request_id': 'req-db6caa5d-6032-46a9-9efc-f6ababb71bfd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.921485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb59ae6d8a24c119a6c7ffde922ed83', 'request_id': 'req-9c9d8263-efa5-4942-8d4c-36b29035113f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.923424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83a714b488774bc5a96530181a69e73a', 'request_id': 'req-db6caa5d-6032-46a9-9efc-f6ababb71bfd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.929311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb59ae6d8a24c119a6c7ffde922ed83', 'request_id': 'req-9c9d8263-efa5-4942-8d4c-36b29035113f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.931422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83a714b488774bc5a96530181a69e73a', 'request_id': 'req-db6caa5d-6032-46a9-9efc-f6ababb71bfd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.937148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb59ae6d8a24c119a6c7ffde922ed83', 'request_id': 'req-9c9d8263-efa5-4942-8d4c-36b29035113f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.945247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] The token cache is being invalidate because project 83a714b488774bc5a96530181a69e73a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:11.945532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.946253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.948417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 951/1875] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:11 2022] DELETE /identity/v3/projects/83a714b488774bc5a96530181a69e73a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:11.949706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:11.951061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:11.952164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] The token cache is being invalidate because project 9fb59ae6d8a24c119a6c7ffde922ed83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:11.952982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.953925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9c9d8263-efa5-4942-8d4c-36b29035113f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:11.955548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 925/1876] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:11 2022] DELETE /identity/v3/projects/9fb59ae6d8a24c119a6c7ffde922ed83 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:11.956772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:11.957074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:11.957349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:11.971590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:11.981888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:11.983254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:12.044739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6caa5d-6032-46a9-9efc-f6ababb71bfd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.060254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 926/1877] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:12.068278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.069964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:12.085629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12501e8e-8d86-46f4-a0d1-290febf39ba8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.093003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.106560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.109374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.109663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] PATH_INFO: `/v3/users/0e44217fd3db4bd2aacea3ff93af0a9b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.111438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e44217fd3db4bd2aacea3ff93af0a9b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.115572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.125888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.126629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.126930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.127199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] PATH_INFO: `/v3/users/b04c6e37ebb7421cba3ff882275118b4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.128525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04c6e37ebb7421cba3ff882275118b4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.131032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.209729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04c6e37ebb7421cba3ff882275118b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.230552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04c6e37ebb7421cba3ff882275118b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.231290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04c6e37ebb7421cba3ff882275118b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.232071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04c6e37ebb7421cba3ff882275118b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.240006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e44217fd3db4bd2aacea3ff93af0a9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.253136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e44217fd3db4bd2aacea3ff93af0a9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.255824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e44217fd3db4bd2aacea3ff93af0a9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.258510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e44217fd3db4bd2aacea3ff93af0a9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.264600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3f1355f6-6277-42c2-a10c-ebf824afd105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04c6e37ebb7421cba3ff882275118b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.283387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c53a1c3-35a7-4e10-943e-3c7b1c06529e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e44217fd3db4bd2aacea3ff93af0a9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:12.284745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 952/1878] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:11 2022] DELETE /identity/v3/users/b04c6e37ebb7421cba3ff882275118b4 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:12.286222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.286569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.286858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.311106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:12.312680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:12.316346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 927/1879] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:12 2022] DELETE /identity/v3/users/0e44217fd3db4bd2aacea3ff93af0a9b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:12.318076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.318383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.318704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.348075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:12.349870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:12.372367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:12.422467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:12.464750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 928/1880] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:12.478609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.480488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:12.522995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 953/1881] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:12.534381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.555331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708bc68-e442-4ecf-82a9-3bc91610b096 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.563741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.564580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.564925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.565317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.566052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:12.566896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.569555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.586572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3c337-91f1-4fd1-869a-e2da3e1cbca1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.588990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 954/1882] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:12.591127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.591749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.592028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.592303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.594217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.603260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.605696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.608777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617de68b-d905-4ffd-b904-755d1f9b2ed1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.615234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.616457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.616930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.617401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.617730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 929/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:12.618413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.621250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.670069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 955/1884] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:12.673828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.708931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6ce22-8738-47dd-a85a-fac5b4f05280 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:12.905305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:12.906133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:12.906359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:12.906660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:12.908318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:12.930616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:12.983571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:12.985502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.006039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8880fbe-9837-49c3-9dee-0f3ad972d308 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.015890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 930/1885] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:12 2022] POST /identity/v3/users => generated 294 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:13.018723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.026350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.027850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.028328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.028596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] PATH_INFO: `/v3/projects/0d3845d063e04633afc306ecb3637a31` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.030461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3845d063e04633afc306ecb3637a31)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.032664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.043373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c86d0-978d-47fb-b887-6e1bcaae91a6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.049740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.050362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.050672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.051493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.051784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.053647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.072170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 931/1886] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:13.085517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.092828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3845d063e04633afc306ecb3637a31', 'request_id': 'req-2177ea7f-ceec-4438-9204-c156e91fd1d4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.102505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cd150-ed1b-40a2-a284-a15b628242b2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.102781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3845d063e04633afc306ecb3637a31', 'request_id': 'req-2177ea7f-ceec-4438-9204-c156e91fd1d4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.126655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3845d063e04633afc306ecb3637a31', 'request_id': 'req-2177ea7f-ceec-4438-9204-c156e91fd1d4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.146739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] The token cache is being invalidate because project 0d3845d063e04633afc306ecb3637a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:13.147090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.148202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2177ea7f-ceec-4438-9204-c156e91fd1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.153400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 956/1887] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:12 2022] DELETE /identity/v3/projects/0d3845d063e04633afc306ecb3637a31 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:13.160610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef415-7c13-43e4-8d84-915b031718c8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.160900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef415-7c13-43e4-8d84-915b031718c8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.161173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfef415-7c13-43e4-8d84-915b031718c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.162393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:13.192463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6cfef415-7c13-43e4-8d84-915b031718c8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:13.193869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6cfef415-7c13-43e4-8d84-915b031718c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.194408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.195264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.195718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.196144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.196948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.202612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.207596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 957/1888] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:13.214520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 932/1889] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:13.218685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.219655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.220164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.220514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.224797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.240260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253dd3a-f94c-487f-994c-e007782548f6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.247047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.247344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:13.247731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.248072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.248426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] PATH_INFO: `/v3/projects/63d1d028aeef412798537d4d23c9041f/users/615ce3fbf38d4cd2836c8e89bb9f9037/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.249696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.263661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 958/1890] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:13.267815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.270948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d1d028aeef412798537d4d23c9041f, user_id=615ce3fbf38d4cd2836c8e89bb9f9037, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.274627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203b663-e621-44e7-ac30-3c7b9876068d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.279984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b77a7-f7ac-4e83-98ef-0a915460b6bf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.289724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.290646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.290862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.291239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.291953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:13.292872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.294745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 933/1891] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:13 2022] PUT /identity/v3/projects/63d1d028aeef412798537d4d23c9041f/users/615ce3fbf38d4cd2836c8e89bb9f9037/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:13.295246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.306854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.307256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.309186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.314957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 959/1892] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:25:13.316079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.328039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd18db-4c33-4625-9d3d-8d7a5e6534ba admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.347713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] MFA Rules not processed for user `615ce3fbf38d4cd2836c8e89bb9f9037`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:13.349563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.412434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.413059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.413334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.413603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.414238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.416192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.422528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:13.440180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 934/1893] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:13.441053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.444072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.457507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 960/1894] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:13.458495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.462194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43c127-b5ca-4f25-80fd-02f735db3d37 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.527768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.528398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.528727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.528999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] PATH_INFO: `/v3/projects/a5a2ceb109e54ff099b467ff034af784` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.529293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891b74f-5b31-4998-beb8-0c39ce9ce727 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.530263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a2ceb109e54ff099b467ff034af784)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.532323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.535733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.536378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.536631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.537049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.538086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.540551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.594391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 961/1895] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:13.595415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.596589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:13.597976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a2ceb109e54ff099b467ff034af784', 'request_id': 'req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.606887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a2ceb109e54ff099b467ff034af784', 'request_id': 'req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.620946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a2ceb109e54ff099b467ff034af784', 'request_id': 'req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.633986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] The token cache is being invalidate because project a5a2ceb109e54ff099b467ff034af784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:13.634394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.635534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.637290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 935/1896] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:13 2022] DELETE /identity/v3/projects/a5a2ceb109e54ff099b467ff034af784 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:13.640534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.641138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.672820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d4d70-b66f-459f-8c5f-058456e8c973 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.677638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.678337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.678718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.679044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.680081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.681748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.708235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 962/1897] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:13.709177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.710837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.725825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e4e4f-3a8e-4869-8068-ebe1f560ff1d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.730066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dc704-4814-44d7-8de8-f7517df6af06 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.739654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.740276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.740547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.740812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.740904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.741447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.741749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.742146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.742513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] PATH_INFO: `/v3/users/fb68250f370d48eab22b602b98c8cdae` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.743409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.743865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb68250f370d48eab22b602b98c8cdae)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.746711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.778384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 936/1898] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:13 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:13.779500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.781112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.799503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bfaa3-9352-4633-92b1-d85d35675469 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.805004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.816653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.816653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.816653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.816653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.816653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.821386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb68250f370d48eab22b602b98c8cdae'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.829135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 937/1899] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:13.830700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.831191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb68250f370d48eab22b602b98c8cdae'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.832177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb68250f370d48eab22b602b98c8cdae'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.833025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb68250f370d48eab22b602b98c8cdae'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.835523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:13.842440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86c53a-467a-435e-b9b4-bf09a03f273f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.848000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.848681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.849024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.849364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.850077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.851776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.855907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb68250f370d48eab22b602b98c8cdae'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:13.868353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 938/1900] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:13.868353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.869668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.871119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 963/1901] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:13 2022] DELETE /identity/v3/users/fb68250f370d48eab22b602b98c8cdae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:13.871932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.873293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:13.883535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab083d1-9a8f-46c5-b6a4-cf62f4f151c8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.960590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3754b-df4d-4822-80b3-c422cc6ce3f7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:13.966053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.967036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.967319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.967593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] PATH_INFO: `/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157/users/9cd80bfe3e8b4e808b1758f9a33c0277/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.968030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:13.968748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:13.969054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:13.969393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:13.970234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:13.972269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:13.986148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 939/1902] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:13.987433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:13.999545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8d77aadc034347b44fd1aa2a4cc157, user_id=9cd80bfe3e8b4e808b1758f9a33c0277, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.001308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef89ed6-e29c-44d1-879c-79f060710e6a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.002398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.007195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.007913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.008487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.008766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] PATH_INFO: `/v3/projects/de5d17d6e9cf42edb66c9f66fc1e360d/users/52c75904c57745ea8b2b85b56da94b41/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.024283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 964/1903] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:13 2022] PUT /identity/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157/users/9cd80bfe3e8b4e808b1758f9a33c0277/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:14.028568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:14.032723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5d17d6e9cf42edb66c9f66fc1e360d, user_id=52c75904c57745ea8b2b85b56da94b41, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.036715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d0f39-fa94-4363-820d-efbb4554f441 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.061366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 940/1904] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:13 2022] PUT /identity/v3/projects/de5d17d6e9cf42edb66c9f66fc1e360d/users/52c75904c57745ea8b2b85b56da94b41/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:14.076279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.076577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.076855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.111447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:14.116244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] MFA Rules not processed for user `52c75904c57745ea8b2b85b56da94b41`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:14.118707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.161403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42f3c-c590-4b22-b7a2-33583c6d8df1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.243009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 941/1905] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:14.248501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.249426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.249589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.249748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.250528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.252188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.255898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:14.257565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.259086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:14.267265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 965/1906] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:14.272796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:14.287980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33024502-a480-406e-9625-49c3048112f3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.293803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.294511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.294956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.295240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] PATH_INFO: `/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157/users/9cd80bfe3e8b4e808b1758f9a33c0277/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.297472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8d77aadc034347b44fd1aa2a4cc157, user_id=9cd80bfe3e8b4e808b1758f9a33c0277, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.300398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1ecf7-0750-4e7a-8291-a89a8c78f89f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.322442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 966/1907] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:14 2022] PUT /identity/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157/users/9cd80bfe3e8b4e808b1758f9a33c0277/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:14.325336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.325653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.325931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.359534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] MFA Rules not processed for user `9cd80bfe3e8b4e808b1758f9a33c0277`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:14.361524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.411501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b769bc-7a30-4034-a9d1-f4c291024271 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.493274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.493973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.494323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.494697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.528909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 967/1908] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:14.530721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:14.534099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.578213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:14.581954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.583588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.596646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 942/1909] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:14.597755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:14.604174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.621170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4b626-c751-412f-b2f1-cf2161a7ea5f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.632976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.634018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.634426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.634819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] PATH_INFO: `/v3/projects/cf196a9b85764eb0a25d8329a1114fb5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.636727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf196a9b85764eb0a25d8329a1114fb5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.639126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.687761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89fddb-99eb-4aa9-af5f-7d82019b4be7 service glance] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:14.697423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf196a9b85764eb0a25d8329a1114fb5', 'request_id': 'req-c63d4174-547a-4bf5-92bf-aa75f5718cca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:14.705683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf196a9b85764eb0a25d8329a1114fb5', 'request_id': 'req-c63d4174-547a-4bf5-92bf-aa75f5718cca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:14.716559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf196a9b85764eb0a25d8329a1114fb5', 'request_id': 'req-c63d4174-547a-4bf5-92bf-aa75f5718cca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:14.732324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] The token cache is being invalidate because project cf196a9b85764eb0a25d8329a1114fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:14.732875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:14.733414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c63d4174-547a-4bf5-92bf-aa75f5718cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:14.736743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 968/1910] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:14 2022] DELETE /identity/v3/projects/cf196a9b85764eb0a25d8329a1114fb5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:14.741838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138829-7bf8-4e6f-999a-fc45f1b66bca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.742996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138829-7bf8-4e6f-999a-fc45f1b66bca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.742996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138829-7bf8-4e6f-999a-fc45f1b66bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.762771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:14.774132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:14.774704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.774969 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.775239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.777578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.789513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9b138829-7bf8-4e6f-999a-fc45f1b66bca None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:14.791512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b138829-7bf8-4e6f-999a-fc45f1b66bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.921190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:14.921592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 969/1911] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:14.922825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:14.931906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:14.932201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:14.932477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:14.952187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 943/1912] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:14.983412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:14.984941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:14.997239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 970/1913] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:14.998324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.010936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.016181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b155d-b586-40c1-9f4a-759a2709f116 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.017686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.030184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.031010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.031315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.031688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.034050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:15.034371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.036009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.049713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc693b-fa1f-407f-be3c-f21d6dcdcc0b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.058651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 944/1914] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:15.060035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.061006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.061289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.061564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] PATH_INFO: `/v3/users/615ce3fbf38d4cd2836c8e89bb9f9037` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.066602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615ce3fbf38d4cd2836c8e89bb9f9037)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.066805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.066903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.085033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98bab5-e782-4e95-9e7a-7d5595d59e22 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.091280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.091900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.092178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.092594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.093236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.095290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.144571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 945/1915] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:15 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:15.147507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.180661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615ce3fbf38d4cd2836c8e89bb9f9037'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:15.193960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615ce3fbf38d4cd2836c8e89bb9f9037'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:15.194858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615ce3fbf38d4cd2836c8e89bb9f9037'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:15.195643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615ce3fbf38d4cd2836c8e89bb9f9037'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:15.213633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:15.218757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7eee7912-80a8-4bdc-85b2-d5203ebc9a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615ce3fbf38d4cd2836c8e89bb9f9037'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:15.241746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 971/1916] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:15 2022] DELETE /identity/v3/users/615ce3fbf38d4cd2836c8e89bb9f9037 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:15.250222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.252773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.253196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.280668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:15.281890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.292961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53c698-b1c9-4b30-ae9c-2b6746fb1357 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.433320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.434707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.435092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.435465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.436838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.442291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.443956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 972/1917] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:15.475247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.476932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.490132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 946/1918] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:15 2022] POST /identity/v3/users => generated 291 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:15.494215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.496377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:15.497868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.511810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd2431-d73e-4bfc-b102-40a6bf62923b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.516956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.517618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.517903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.518182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.519041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.520957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.532403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 947/1919] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:15.536441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.547789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3ce1f-c9be-4d4a-9816-80843b398a05 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.554775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.555392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.555513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557fa1-6cf3-44e5-8486-1e8284ff4044 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.555743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.556024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.556667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.558385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.559998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.560810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.561091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.561357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.571494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 948/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:15.575924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.576211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.577546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e560696-4a71-45dd-abdc-d542bf882853 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.588688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 973/1921] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:15.601618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d10097-ce6c-40e8-99fc-3432c6f09e9d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.607803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.608610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.609004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.609426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] PATH_INFO: `/v3/projects/87a39465f92b47ce913c26c1bb2ce146/users/808c84fbf9704967b0cc67cbc0104243/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.626117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a39465f92b47ce913c26c1bb2ce146, user_id=808c84fbf9704967b0cc67cbc0104243, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.629194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.649946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 949/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:15 2022] PUT /identity/v3/projects/87a39465f92b47ce913c26c1bb2ce146/users/808c84fbf9704967b0cc67cbc0104243/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:15.660983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.661270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.661543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.697115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] MFA Rules not processed for user `808c84fbf9704967b0cc67cbc0104243`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:15.699496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.780623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 974/1923] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:15.802417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.804924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.894041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c6d2c-1bb1-428d-9b0f-551f6569fc8e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:15.900619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:15.900619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:15.900619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:15.900619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:15.910413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:15.911891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:15.921971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 950/1924] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:15.949163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.970989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:15.972772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:15.995747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:16.037427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c2b3c-ca98-4eb4-b986-90a49c9377c5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.042132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:16.043317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.043591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.043859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] PATH_INFO: `/v3/users/808c84fbf9704967b0cc67cbc0104243` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.045021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=808c84fbf9704967b0cc67cbc0104243)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:16.047000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:16.051355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c21c6-00f6-459f-8e27-0861d022ef0a service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.056376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:16.057088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.057439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.057822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] PATH_INFO: `/v3/projects/63d1d028aeef412798537d4d23c9041f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.061578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d1d028aeef412798537d4d23c9041f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:16.063681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:16.097915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:16.100966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808c84fbf9704967b0cc67cbc0104243'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.108849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808c84fbf9704967b0cc67cbc0104243'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.109472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808c84fbf9704967b0cc67cbc0104243'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.110231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808c84fbf9704967b0cc67cbc0104243'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.112560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d1d028aeef412798537d4d23c9041f', 'request_id': 'req-120910be-40f9-4a22-abb0-083c641b5637'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.119350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d1d028aeef412798537d4d23c9041f', 'request_id': 'req-120910be-40f9-4a22-abb0-083c641b5637'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.122761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0c469c22-4b6e-452a-bc4f-0f5107cd52cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808c84fbf9704967b0cc67cbc0104243'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.127374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d1d028aeef412798537d4d23c9041f', 'request_id': 'req-120910be-40f9-4a22-abb0-083c641b5637'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.138183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 975/1925] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:15 2022] DELETE /identity/v3/users/808c84fbf9704967b0cc67cbc0104243 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:16.143257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.143634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.144004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.155245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] The token cache is being invalidate because project 63d1d028aeef412798537d4d23c9041f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:16.155538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.156234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.157798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 951/1926] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:15 2022] DELETE /identity/v3/projects/63d1d028aeef412798537d4d23c9041f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:16.161735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:16.163129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.165416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:16.166619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.234908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120910be-40f9-4a22-abb0-083c641b5637 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.238044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 976/1927] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:16.239928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:16.247830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:16.249898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.250851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.251199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.251575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] PATH_INFO: `/v3/users/9cd80bfe3e8b4e808b1758f9a33c0277` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.252873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cd80bfe3e8b4e808b1758f9a33c0277)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:16.254981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:16.306913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cd80bfe3e8b4e808b1758f9a33c0277'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.309991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:16.316705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cd80bfe3e8b4e808b1758f9a33c0277'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.317247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78dcd3-ad0e-46d1-ad4e-2a539d06d3c0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.317667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cd80bfe3e8b4e808b1758f9a33c0277'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.318597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cd80bfe3e8b4e808b1758f9a33c0277'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.321555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:16.322245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.322686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.322918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.325064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.326758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cd80bfe3e8b4e808b1758f9a33c0277'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.337544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:16.339061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:16.347503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 952/1928] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:16 2022] DELETE /identity/v3/users/9cd80bfe3e8b4e808b1758f9a33c0277 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:16.352461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 977/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:16.721153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:16.721153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.809664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abf772-81d4-4b1f-be3d-f8112e3403a6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.817295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:16.818050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.819404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.819789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] PATH_INFO: `/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:16.821264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8d77aadc034347b44fd1aa2a4cc157)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:16.824752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:16.894537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8d77aadc034347b44fd1aa2a4cc157', 'request_id': 'req-7333d96d-4863-41d9-94cf-7671d9e532d1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.932828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8d77aadc034347b44fd1aa2a4cc157', 'request_id': 'req-7333d96d-4863-41d9-94cf-7671d9e532d1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.945802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:16.947836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:16.956258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8d77aadc034347b44fd1aa2a4cc157', 'request_id': 'req-7333d96d-4863-41d9-94cf-7671d9e532d1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.972209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] The token cache is being invalidate because project fb8d77aadc034347b44fd1aa2a4cc157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:16.972348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d92c-0a1a-49f4-9b82-4275ed37ff8b service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:16.972500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.973336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7333d96d-4863-41d9-94cf-7671d9e532d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:16.975533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 953/1930] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:16 2022] DELETE /identity/v3/projects/fb8d77aadc034347b44fd1aa2a4cc157 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:16.981906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc698fa1-af44-4c13-8f38-2e2816baa44d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:16.982185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc698fa1-af44-4c13-8f38-2e2816baa44d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:16.982454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc698fa1-af44-4c13-8f38-2e2816baa44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.028009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cc698fa1-af44-4c13-8f38-2e2816baa44d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.029394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cc698fa1-af44-4c13-8f38-2e2816baa44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.055026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.055673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.056086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.056370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] PATH_INFO: `/v3/projects/87a39465f92b47ce913c26c1bb2ce146` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.057520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a39465f92b47ce913c26c1bb2ce146)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.059721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.067312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:17.108854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:17.128203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 954/1931] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:17.136858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.137142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.137416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.140081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a39465f92b47ce913c26c1bb2ce146', 'request_id': 'req-0a468875-52da-4c9b-82d7-539dae19f90d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:17.160132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a39465f92b47ce913c26c1bb2ce146', 'request_id': 'req-0a468875-52da-4c9b-82d7-539dae19f90d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:17.176711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a39465f92b47ce913c26c1bb2ce146', 'request_id': 'req-0a468875-52da-4c9b-82d7-539dae19f90d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:17.181088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.183539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.196925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] The token cache is being invalidate because project 87a39465f92b47ce913c26c1bb2ce146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:17.197840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:17.199247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0a468875-52da-4c9b-82d7-539dae19f90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:17.203939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 978/1932] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:16 2022] DELETE /identity/v3/projects/87a39465f92b47ce913c26c1bb2ce146 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:17.207359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b950-e96b-4012-9ad1-509202161aed None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.207649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b950-e96b-4012-9ad1-509202161aed None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.207923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3b950-e96b-4012-9ad1-509202161aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.246966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b6d3b950-e96b-4012-9ad1-509202161aed None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.248120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6d3b950-e96b-4012-9ad1-509202161aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.292845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 955/1933] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:17.296976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.311154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a5f39-83eb-45c8-9f7a-88bc9ef12c87 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.317117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.317727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.318003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.318271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.318977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:17.319273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.320179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 979/1934] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:17.324111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.329657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.329970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.330245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.339456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 956/1935] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:25:17.340391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.350228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.350397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba079a8-e44c-4f97-acef-276609de7179 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.351487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.355017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.355613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.355896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.356164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.356784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.358692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.359590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 980/1936] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:17.363649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.373218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5de2f-7c95-4fd0-8925-36e8769aef72 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.378313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.379191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.379639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.379949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.380748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:17.381046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.382836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.393158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 957/1937] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:17.394114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.399225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 981/1938] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:25:17.401995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.418990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2d06b-0861-4611-a931-c77ab2814496 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.469512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca70b3-f5f8-45e0-b4ea-0058524b26a9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.474498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.475557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.475851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.476128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.476778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.478691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.492895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.493720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.494161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.494617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.495907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.498030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.504286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 958/1939] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:17.506148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.521760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db53df3-b3cc-4fb7-89b2-70478baa34b6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.532578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.532770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:17.533872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.534332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.534835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.536862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.540120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.549058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 982/1940] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:17.552925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.557204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 959/1941] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:17.561126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.575797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f088e-01dd-4406-a347-675db40ecd8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.626287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:17.636425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab580d-9b4a-49d2-ac2e-30fe579800e5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.641368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.642040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.642321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.642604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.643523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.645195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.649210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.650118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.650524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.650947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.652073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.654569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.669708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 983/1942] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:17.674289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.680419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 960/1943] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:17.685546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.689741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2921d-187a-4c5a-9b90-59a268b7be30 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.694903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.695516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.695797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.696060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.696714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.698308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba19-ab2b-41a6-a1ee-86f475c87609 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.699047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.703557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.704732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.705246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.705763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] PATH_INFO: `/v3/projects/70af77f3ec074d45b6ce863a6dce70bb/users/12aa599944f94c93a2bbea50c76146dd/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.711664 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 984/1944] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:17.715923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.722856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70af77f3ec074d45b6ce863a6dce70bb, user_id=12aa599944f94c93a2bbea50c76146dd, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.725519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91698e11-42cf-4fab-b7c4-5eebf8db9311 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.726172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63cca-ae8b-46c1-8f1b-e4d7d687842f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.731863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.732488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.732763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.733019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.733653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.735426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.740823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 961/1945] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:17 2022] PUT /identity/v3/projects/70af77f3ec074d45b6ce863a6dce70bb/users/12aa599944f94c93a2bbea50c76146dd/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:17.751745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 985/1946] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:17.754597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.754964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.755391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.756683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.773340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43455f6d-169c-4dd7-96a0-0be069bc25ac admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.790932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] MFA Rules not processed for user `12aa599944f94c93a2bbea50c76146dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.792742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.851916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:17.852615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.852900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.853178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] PATH_INFO: `/v3/projects/7384e765a88f428bb8e936be67884995/users/5120800fff3e40f8b685cf933aaff03c/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.875151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7384e765a88f428bb8e936be67884995, user_id=5120800fff3e40f8b685cf933aaff03c, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:17.877902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf3826-8184-4673-a14a-5dfc4f53825d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:17.896338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 986/1947] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:17 2022] PUT /identity/v3/projects/7384e765a88f428bb8e936be67884995/users/5120800fff3e40f8b685cf933aaff03c/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:17.906881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 962/1948] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:17.907113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:17.907230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:17.907501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:17.911046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:17.927109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b298d-5e1f-471d-827c-a5cbd33f8a09 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:17.940956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] MFA Rules not processed for user `5120800fff3e40f8b685cf933aaff03c`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:17.942197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:17.986917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:18.007152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.008288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.008288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.008462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.009066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.011090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.017325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 987/1949] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:18.021510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.036973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab4ba8-8e0c-45be-9797-be14b4b8ca5f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.041644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.042315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.042610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.043133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.043260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 963/1950] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:17 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:18.043593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.046119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.047267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.085046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 988/1951] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:18.089995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.127436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:18.177107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f6086-46e4-4923-a669-fb7e53450026 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.195961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.196725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.197030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.197352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.198097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.201312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.215210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8d19b-a831-42fd-9c18-d0e1d992b789 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.221470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.223085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.223085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.223085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.223600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.226065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.255997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 989/1952] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:18.257274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.264126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 964/1953] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:18.269948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.280125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774734-5142-4861-a122-7343bef96c8d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.285250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.286769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.287049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.287319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.288620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.290365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.307298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 990/1954] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:18.310237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.321960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08a3f-4fbc-434a-9baa-5f4e3f417476 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.326869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.326989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f12bb-568b-4f2c-9d57-1bbd2aeedc09 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.327455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.327731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.328125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.328821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.330420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.333590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.334297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.334782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.335117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.335939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.338335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.344870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 965/1955] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:18.349438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.356438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 991/1956] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:18.360703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.360855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65eb4ca-a125-4bf2-8966-60bd0d2fe979 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.366605 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.367419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.367801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.368164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] PATH_INFO: `/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64/users/2b5e790f6c934a76bcc680edf494a272/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.373837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63c755-4709-449a-b587-99b4cefcf2eb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.381846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.382516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.382818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.383346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] PATH_INFO: `/v3/projects/46f1539694bb4466a2419160c3055f7f/users/b20718d2944c414ca87d5efd808d4ed3/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.388103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d9fc647ba44771ad8ca0cd810f9b64, user_id=2b5e790f6c934a76bcc680edf494a272, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.394309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.401892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f1539694bb4466a2419160c3055f7f, user_id=b20718d2944c414ca87d5efd808d4ed3, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.414966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22272d-4468-49f5-a9ec-7f8396b0d030 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.426928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 966/1957] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:18 2022] PUT /identity/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64/users/2b5e790f6c934a76bcc680edf494a272/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:18.432625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 992/1958] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:18 2022] PUT /identity/v3/projects/46f1539694bb4466a2419160c3055f7f/users/b20718d2944c414ca87d5efd808d4ed3/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:18.442369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.443143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.443609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.444039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.491002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] MFA Rules not processed for user `b20718d2944c414ca87d5efd808d4ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:18.492889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:18.559739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:18.594702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 993/1959] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:18.594902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d135e2-59f3-4b64-85f1-dbabc656dc4d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.596621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.684963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae20b-64e9-44fa-816c-0c26ce4a37a6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.691400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.692028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.692323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.692710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.693864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.704224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.704541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.705171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.705528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.705801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.706449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.712114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.735368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 967/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:18.737211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.751859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 994/1961] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:18.756415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.766286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b720c3-7ea2-4460-9702-6802a2ee64e6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.805119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:18.854891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f443e5-55b0-4845-8889-ce0daaf373fc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.860215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.860854 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.861112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.861507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.862149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.865174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.865516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:18.866548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.867054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.867382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] PATH_INFO: `/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64/users/2b5e790f6c934a76bcc680edf494a272/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.869484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d9fc647ba44771ad8ca0cd810f9b64, user_id=2b5e790f6c934a76bcc680edf494a272, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:18.878884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e936d8-02b2-4604-bc73-881b1924d050 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:18.890686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 968/1962] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:18 2022] PUT /identity/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64/users/2b5e790f6c934a76bcc680edf494a272/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:18.898526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 995/1963] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:18.898735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:18.898735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:18.899912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:18.903046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:18.914887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ad02b-ab10-42a9-8ecf-02397beab2dc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:18.942135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] MFA Rules not processed for user `2b5e790f6c934a76bcc680edf494a272`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:18.944396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:19.020380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.021140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.021594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.021892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.022793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.026225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.042682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 996/1964] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:19.045975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.051697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:19.059909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fba3d7-479d-43f6-879b-390078a683f3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.066880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.067691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.068101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.068720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] PATH_INFO: `/v3/projects/6ba8e3f6167a4f569fb54131872d93e4/users/3f555a4d24f144ce91d20b3ecad31036/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.097782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba8e3f6167a4f569fb54131872d93e4, user_id=3f555a4d24f144ce91d20b3ecad31036, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.100684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.121199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 997/1965] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:19 2022] PUT /identity/v3/projects/6ba8e3f6167a4f569fb54131872d93e4/users/3f555a4d24f144ce91d20b3ecad31036/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:19.123406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.127451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 969/1966] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:19.139858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.142092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:19.199562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54811b-8555-4c7d-9ca3-293d786af206 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.205147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.206208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.206486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.206789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.207446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.209377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.223229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 998/1967] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:19.224237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.232923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ddf1e-1e18-4a8c-b5ff-15da45c38adf None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.235569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09114ce3-e999-4db5-b0f6-027a389a82d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.241614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.242283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.243033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.243313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] PATH_INFO: `/v3/projects/6ba8e3f6167a4f569fb54131872d93e4/users/3f555a4d24f144ce91d20b3ecad31036/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.245624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba8e3f6167a4f569fb54131872d93e4, user_id=3f555a4d24f144ce91d20b3ecad31036, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.248427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4aa11-2487-4ecf-b8e0-29af42609af4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.261532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 999/1968] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:19 2022] PUT /identity/v3/projects/6ba8e3f6167a4f569fb54131872d93e4/users/3f555a4d24f144ce91d20b3ecad31036/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:19.272776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.273059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.273329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.294618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.296863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.297209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.297561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.308078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] MFA Rules not processed for user `3f555a4d24f144ce91d20b3ecad31036`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:19.309477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:19.318041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:19.408479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.410770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.423684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 970/1969] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:19.443478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1000/1970] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:19.535655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.545146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34677c5-7789-4fb9-9181-b3a42e3cf534 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.604268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.605486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.605761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.606022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:19.672258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:19.673688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:19.683658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 971/1971] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:19.912762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:19.915157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:19.929417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba43b3-113f-4e8d-9a68-5809f3b69d27 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:19.933539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:19.934379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:19.934738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:19.935105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:20.005316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:20.024228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:20.025769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:20.037062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1001/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:21.423826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:21.436159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14515cb-4b9c-47fa-982e-0596c874870f service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:21.442884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:21.443942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:21.444418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:21.444887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:21.568856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:21.570557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:21.591332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 972/1973] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:22.419263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:22.427857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1e33-f7ca-4a78-b5f5-ea58f15f28e4 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:22.431597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:22.432229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:22.433142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:22.433439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:22.443249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:22.444626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c03aa4-d42a-46f8-8769-e5b03f96e0ba service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:22.455925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1002/1974] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:22.852445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:22.858563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:22.923383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e0524-9325-47f7-8514-d558162fa9c0 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:22.931309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:22.931948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:22.932224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:22.932497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] PATH_INFO: `/v3/users/12aa599944f94c93a2bbea50c76146dd` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:22.934242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12aa599944f94c93a2bbea50c76146dd)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:22.936261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:22.984858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12aa599944f94c93a2bbea50c76146dd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:22.992261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12aa599944f94c93a2bbea50c76146dd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:22.993076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12aa599944f94c93a2bbea50c76146dd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:22.993695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12aa599944f94c93a2bbea50c76146dd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.001660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12aa599944f94c93a2bbea50c76146dd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.019688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 973/1975] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:22 2022] DELETE /identity/v3/users/12aa599944f94c93a2bbea50c76146dd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:23.024078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:23.024670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:23.024670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:23.045649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:23.047014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:23.120794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1003/1976] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:23.130452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:23.131989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:23.193919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0b1bb-f508-4130-b84b-107a5befc3c1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:23.197704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:23.198896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:23.199167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:23.199430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:23.208788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:23.210269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:23.223729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 974/1977] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:23.756003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:23.758627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:23.786060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e838a1f-7a90-4824-bc8a-e03599221d6c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:23.794144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:23.794899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:23.795362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:23.795648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] PATH_INFO: `/v3/projects/70af77f3ec074d45b6ce863a6dce70bb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:23.796856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70af77f3ec074d45b6ce863a6dce70bb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:23.798766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:23.861440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70af77f3ec074d45b6ce863a6dce70bb', 'request_id': 'req-73af452f-ca15-4976-a03a-cfc4fef7f551'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.868346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70af77f3ec074d45b6ce863a6dce70bb', 'request_id': 'req-73af452f-ca15-4976-a03a-cfc4fef7f551'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.876496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70af77f3ec074d45b6ce863a6dce70bb', 'request_id': 'req-73af452f-ca15-4976-a03a-cfc4fef7f551'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.888395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] The token cache is being invalidate because project 70af77f3ec074d45b6ce863a6dce70bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:23.888689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.889383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:23.891214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1004/1978] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:23 2022] DELETE /identity/v3/projects/70af77f3ec074d45b6ce863a6dce70bb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:23.896518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:23.907353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b32976-ac94-4d15-bd63-a6660f0367a0 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:23.945863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:23.981441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:23.982134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:23.982479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:23.982812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] PATH_INFO: `/v3/users/2b5e790f6c934a76bcc680edf494a272` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:23.984288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b5e790f6c934a76bcc680edf494a272)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:23.986444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.032255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b5e790f6c934a76bcc680edf494a272'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.039247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b5e790f6c934a76bcc680edf494a272'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.039885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b5e790f6c934a76bcc680edf494a272'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.040481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b5e790f6c934a76bcc680edf494a272'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.047767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-08c7cb43-a06f-4189-aa9f-03c521a2d607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b5e790f6c934a76bcc680edf494a272'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.065086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 975/1979] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:23 2022] DELETE /identity/v3/users/2b5e790f6c934a76bcc680edf494a272 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:24.350744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.352992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:24.371891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af452f-ca15-4976-a03a-cfc4fef7f551 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.429235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:24.437313 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.438417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.438871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.439254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] PATH_INFO: `/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.440987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d9fc647ba44771ad8ca0cd810f9b64)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.443128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.498222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d9fc647ba44771ad8ca0cd810f9b64', 'request_id': 'req-8c93825d-a003-4fac-900b-ef600437b27d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.504898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d9fc647ba44771ad8ca0cd810f9b64', 'request_id': 'req-8c93825d-a003-4fac-900b-ef600437b27d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.514720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d9fc647ba44771ad8ca0cd810f9b64', 'request_id': 'req-8c93825d-a003-4fac-900b-ef600437b27d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.525834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] The token cache is being invalidate because project f7d9fc647ba44771ad8ca0cd810f9b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:24.526957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.527604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c93825d-a003-4fac-900b-ef600437b27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:24.529124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1005/1980] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:24 2022] DELETE /identity/v3/projects/f7d9fc647ba44771ad8ca0cd810f9b64 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:24.534091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.534382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.534696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.555544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:24.556764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:24.617075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 976/1981] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:24.624990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.625302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.625663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.646166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:24.647403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:24.658498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1006/1982] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:24.660883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.672559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c1c22-cfec-4183-9d7e-cf725cb48a30 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.677957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.678562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.678863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.679317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.679978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:24.680353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.682593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.697358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 977/1983] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:24.701750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.712159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafe953-ebe5-40a3-9628-30bed68d2758 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.717066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.717670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.717947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.718215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.718897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.720794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.751901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1007/1984] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:24 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:24.753033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.763326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca77f-56ce-420f-aadc-d6754106f6ea admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.797312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:24.837489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.837489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.837489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.837489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.837936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.839668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.862434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 978/1985] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:24 2022] POST /identity/v3/users => generated 305 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:24.866121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.876872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2444-1770-4bde-8766-7947be48777c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.881637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.882162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.882457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.882746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.883757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.885454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.897009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1008/1986] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:24.900803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.911650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19960-1bc4-4650-9dfe-72e32199b110 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.916305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.917580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.917867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.918139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.918871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.920474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab5374-5339-4189-b60f-abd06c1d2884 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.931028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 979/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:24.934800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:24.946465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1e496-a1b9-4b7c-b634-41de99c56736 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:24.951344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:24.952132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:24.952511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:24.953609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] PATH_INFO: `/v3/projects/c4f0ac1be49c4e5dafb42d2a35ae45fa/users/de11bdf931b245a9a8b519808fdc1d3c/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:24.972172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f0ac1be49c4e5dafb42d2a35ae45fa, user_id=de11bdf931b245a9a8b519808fdc1d3c, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:24.974835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:24.988946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1009/1988] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:24 2022] PUT /identity/v3/projects/c4f0ac1be49c4e5dafb42d2a35ae45fa/users/de11bdf931b245a9a8b519808fdc1d3c/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:25.002550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.002875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.003146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.031893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] MFA Rules not processed for user `de11bdf931b245a9a8b519808fdc1d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:25.033146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:25.121863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 980/1989] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:25 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:25.126134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.206796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c56d9-9012-4c76-8b89-ec77b2f52b53 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.211859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.212549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.212829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.213259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.214334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.216551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.250050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1010/1990] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:25 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:25.251337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.263746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba7231-34c7-4ebe-bd0a-1c4c16a2f628 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.328747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.329350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.329682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.329957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.330594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.332512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.355352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 981/1991] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:25 2022] POST /identity/v3/users => generated 310 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:25.359376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.370107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d8d19-aa67-4209-94fe-921189648f70 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.374720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.375358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.375602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.375871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.376509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.378114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.379381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:25.391703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1011/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:25.395557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.406379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a4e94-5b20-4db0-ba32-816e782e3686 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.411088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.411776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.412074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.412347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] PATH_INFO: `/v3/projects/0955fcc17e704b17939c0a8c0100e425/users/206e440512ce4a8d88ccf1454a32eee9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.431374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0955fcc17e704b17939c0a8c0100e425, user_id=206e440512ce4a8d88ccf1454a32eee9, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.434243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.448388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 982/1993] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:25 2022] PUT /identity/v3/projects/0955fcc17e704b17939c0a8c0100e425/users/206e440512ce4a8d88ccf1454a32eee9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:25.452431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.462538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce226329-6b0a-4edf-8513-e313bd7aaf72 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.528206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.528827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.529105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.529371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.530007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.531635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3167be-0853-429d-847b-40610ced750a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.541992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1012/1994] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:25.546123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.556464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f2805-62c0-4550-a15d-a4f0b5827a36 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.561575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.562200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.562475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.562811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] PATH_INFO: `/v3/projects/0955fcc17e704b17939c0a8c0100e425/users/206e440512ce4a8d88ccf1454a32eee9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.564701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0955fcc17e704b17939c0a8c0100e425, user_id=206e440512ce4a8d88ccf1454a32eee9, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.567359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.569507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:25.579558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 983/1995] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:25 2022] PUT /identity/v3/projects/0955fcc17e704b17939c0a8c0100e425/users/206e440512ce4a8d88ccf1454a32eee9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:25.589952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.590239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.590512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.621344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] MFA Rules not processed for user `206e440512ce4a8d88ccf1454a32eee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:25.622444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:25.693372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1013/1996] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:25 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:25.704694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.706535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:25.766261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05405b-0214-4186-b3f8-4f65197520e5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.770097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.770742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.771025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.771311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:25.781229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:25.782721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:25.794050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 984/1997] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:25.966169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:25.977278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71c399-f201-47f4-906a-d64dead5f5c8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:25.981793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:25.982409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:25.982726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:25.983018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:26.039598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:26.041027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5c173-81ef-40cb-ba6d-26f594f35492 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:26.050406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1014/1998] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:25 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:26.448532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:26.450222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:26.514799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc5c47-6ee1-472e-b9e1-f68f5ebb025c service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:26.519223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:26.519841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:26.520116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:26.520386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] PATH_INFO: `/v3/users/5120800fff3e40f8b685cf933aaff03c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:26.521479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5120800fff3e40f8b685cf933aaff03c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:26.523403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:26.573917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5120800fff3e40f8b685cf933aaff03c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:26.581666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5120800fff3e40f8b685cf933aaff03c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:26.582563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5120800fff3e40f8b685cf933aaff03c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:26.583019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5120800fff3e40f8b685cf933aaff03c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:26.590548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5120800fff3e40f8b685cf933aaff03c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:26.608363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 985/1999] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:26 2022] DELETE /identity/v3/users/5120800fff3e40f8b685cf933aaff03c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:26.612891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:26.613184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:26.613452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:26.632025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:26.633475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:26.642995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:26.692308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1015/2000] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:26.702547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:26.703998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:26.722695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:26.762061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965b08-f1e4-4eff-b14c-fe5627b4e9de admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:26.765621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:26.766217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:26.766493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:26.766791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:26.775655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:26.777657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:26.786961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 986/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:27.064301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:27.065762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:27.093600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ea959-d4e2-4684-90ac-dae7428e48de None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:27.105561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:27.106620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:27.107046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:27.107323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] PATH_INFO: `/v3/projects/7384e765a88f428bb8e936be67884995` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:27.108495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7384e765a88f428bb8e936be67884995)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:27.110412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:27.170828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7384e765a88f428bb8e936be67884995', 'request_id': 'req-e6bae082-504a-40cd-9cea-59ee625e2edd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.183185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7384e765a88f428bb8e936be67884995', 'request_id': 'req-e6bae082-504a-40cd-9cea-59ee625e2edd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.194175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7384e765a88f428bb8e936be67884995', 'request_id': 'req-e6bae082-504a-40cd-9cea-59ee625e2edd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.206635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] The token cache is being invalidate because project 7384e765a88f428bb8e936be67884995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:27.206963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.207802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.209449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1016/2002] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:27 2022] DELETE /identity/v3/projects/7384e765a88f428bb8e936be67884995 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:27.214087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:27.223800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f934-1afd-497d-acfc-efa60c165c5e service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:27.283580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:27.284325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:27.284606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:27.284875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] PATH_INFO: `/v3/users/b20718d2944c414ca87d5efd808d4ed3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:27.286004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20718d2944c414ca87d5efd808d4ed3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:27.287973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:27.339473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20718d2944c414ca87d5efd808d4ed3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.347574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20718d2944c414ca87d5efd808d4ed3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.348315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20718d2944c414ca87d5efd808d4ed3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.349002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20718d2944c414ca87d5efd808d4ed3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.356464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20718d2944c414ca87d5efd808d4ed3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.374106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 987/2003] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:27 2022] DELETE /identity/v3/users/b20718d2944c414ca87d5efd808d4ed3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:27.759654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:27.761414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:27.785205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bae082-504a-40cd-9cea-59ee625e2edd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:27.859075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:27.859734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:27.860014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:27.860289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] PATH_INFO: `/v3/projects/46f1539694bb4466a2419160c3055f7f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:27.861449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f1539694bb4466a2419160c3055f7f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:27.863413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:27.906600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:27.914759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f1539694bb4466a2419160c3055f7f', 'request_id': 'req-fe7380df-95fd-46a9-9708-b06381d9c741'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.921993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f1539694bb4466a2419160c3055f7f', 'request_id': 'req-fe7380df-95fd-46a9-9708-b06381d9c741'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.929505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f1539694bb4466a2419160c3055f7f', 'request_id': 'req-fe7380df-95fd-46a9-9708-b06381d9c741'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.940025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] The token cache is being invalidate because project 46f1539694bb4466a2419160c3055f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:27.940333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.940923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:27.942487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1017/2004] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:27 2022] DELETE /identity/v3/projects/46f1539694bb4466a2419160c3055f7f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:27.946343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:27.947855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:27.961215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41af7ac-07a4-4b53-9893-49f370aaa0ca admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:27.997634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:28.023691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.024342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.024602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.024872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] PATH_INFO: `/v3/users/3f555a4d24f144ce91d20b3ecad31036` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.026504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f555a4d24f144ce91d20b3ecad31036)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.028504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.072895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f555a4d24f144ce91d20b3ecad31036'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.080014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f555a4d24f144ce91d20b3ecad31036'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.080643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f555a4d24f144ce91d20b3ecad31036'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.081291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f555a4d24f144ce91d20b3ecad31036'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.088837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ffa8a018-01f7-4c23-9b95-686ceed5bac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f555a4d24f144ce91d20b3ecad31036'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.106226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 988/2005] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:27 2022] DELETE /identity/v3/users/3f555a4d24f144ce91d20b3ecad31036 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:28.448670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.450147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:28.463609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7380df-95fd-46a9-9708-b06381d9c741 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.533630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.534292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.534558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.534963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] PATH_INFO: `/v3/projects/6ba8e3f6167a4f569fb54131872d93e4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.536333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba8e3f6167a4f569fb54131872d93e4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.538503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.586452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba8e3f6167a4f569fb54131872d93e4', 'request_id': 'req-c062182e-8d2d-4bcf-a5cf-952d75876394'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.593274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba8e3f6167a4f569fb54131872d93e4', 'request_id': 'req-c062182e-8d2d-4bcf-a5cf-952d75876394'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.600634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba8e3f6167a4f569fb54131872d93e4', 'request_id': 'req-c062182e-8d2d-4bcf-a5cf-952d75876394'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.611284 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] The token cache is being invalidate because project 6ba8e3f6167a4f569fb54131872d93e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:28.611573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.612190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c062182e-8d2d-4bcf-a5cf-952d75876394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:28.613661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1018/2006] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:28 2022] DELETE /identity/v3/projects/6ba8e3f6167a4f569fb54131872d93e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:28.620066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.620355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.620632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.642197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:28.642731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:28.704061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 989/2007] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:28.711874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446da24-e667-4934-b192-6ac756d06425 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.712163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446da24-e667-4934-b192-6ac756d06425 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.712436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446da24-e667-4934-b192-6ac756d06425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.734711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f446da24-e667-4934-b192-6ac756d06425 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:28.735969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f446da24-e667-4934-b192-6ac756d06425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:28.746956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1019/2008] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:28.751404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.761459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c60fd-3a2a-486c-934e-a84337bafc39 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.766052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.766694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.766950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.767214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.767856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:28.768175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.769885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.782157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 990/2009] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:25:28.786394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446da24-e667-4934-b192-6ac756d06425 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.795948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446da24-e667-4934-b192-6ac756d06425 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.800686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.801277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.801553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.801824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.802553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.804503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.834194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1020/2010] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:28 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:28.838393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.850140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849da4f-58f8-4705-85a7-0686bec12988 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.895776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:28.912995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.913598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.913876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.914152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.914809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.917255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.938662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 991/2011] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:28 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:28.942643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.953806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd494c-6706-432d-bf65-e1acb1847469 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.958757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.959435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.959714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.959985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.960617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.962217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:28.972835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1021/2012] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:28.977370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:28.987755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dfa5e-21dd-44b9-a66c-359bbb107431 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:28.992338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:28.993059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:28.993369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:28.993646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:28.994552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:28.996334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d9ad7-90ef-462c-ac00-71f8556264f2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:29.007330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 992/2013] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:29.012880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:29.023696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7eb094-005b-4f13-9f3d-05ef04ae8060 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:29.033133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:29.033956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.034338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.034745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] PATH_INFO: `/v3/projects/21878b013b8a49d7a0aebfd7705faefe/users/28e2ee1e7d8c46d1acf74aa5093a8e12/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.051083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=21878b013b8a49d7a0aebfd7705faefe, user_id=28e2ee1e7d8c46d1acf74aa5093a8e12, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:29.054128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:29.057631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:29.083329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1022/2014] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:29 2022] PUT /identity/v3/projects/21878b013b8a49d7a0aebfd7705faefe/users/28e2ee1e7d8c46d1acf74aa5093a8e12/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:29.095354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.095701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.096114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.125152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] MFA Rules not processed for user `28e2ee1e7d8c46d1acf74aa5093a8e12`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:29.126365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:29.190741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 993/2015] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:29.202351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:29.204085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:29.259554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c8c6f-9a67-4c2b-8e2b-c30c4b4c19fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:29.263752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:29.264353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.264632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.264902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.272945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:29.274366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be79362-8c2a-4797-bb16-ff35df465533 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:29.282992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1023/2016] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:29.308094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:29.372121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41767a7b-0ec6-4f51-8478-6e717469874e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:29.376683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:29.377311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.377583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.377875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] PATH_INFO: `/v3/users/28e2ee1e7d8c46d1acf74aa5093a8e12` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.379024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e2ee1e7d8c46d1acf74aa5093a8e12)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:29.380924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:29.434545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e2ee1e7d8c46d1acf74aa5093a8e12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:29.443814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e2ee1e7d8c46d1acf74aa5093a8e12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:29.444641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e2ee1e7d8c46d1acf74aa5093a8e12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:29.445381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e2ee1e7d8c46d1acf74aa5093a8e12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:29.447119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:29.459324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e2ee1e7d8c46d1acf74aa5093a8e12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:29.478680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 994/2017] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:29 2022] DELETE /identity/v3/users/28e2ee1e7d8c46d1acf74aa5093a8e12 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:29.483604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.484076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.484507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.509104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:29.510733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:29.568614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1024/2018] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:29.578640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:29.580056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:29.641543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041712a-4217-43dd-9fb3-2051794e9055 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:29.645394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:29.646237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.646237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.646509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.655993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:29.657415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a36f0-00c3-440b-9f31-3cdd0ebc4a6a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:29.668794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 995/2019] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:29.943360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:29.944896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:29.957287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53447c2-9bc9-4d6e-950e-ada655b11c07 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:29.961907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:29.962535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:29.962840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:29.963268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] PATH_INFO: `/v3/projects/21878b013b8a49d7a0aebfd7705faefe` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:29.964386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21878b013b8a49d7a0aebfd7705faefe)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:29.966280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.016963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21878b013b8a49d7a0aebfd7705faefe', 'request_id': 'req-1d81678f-9032-4d70-ae85-f81ad6b20228'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:30.023085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21878b013b8a49d7a0aebfd7705faefe', 'request_id': 'req-1d81678f-9032-4d70-ae85-f81ad6b20228'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:30.030923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21878b013b8a49d7a0aebfd7705faefe', 'request_id': 'req-1d81678f-9032-4d70-ae85-f81ad6b20228'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:30.040934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] The token cache is being invalidate because project 21878b013b8a49d7a0aebfd7705faefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:30.041218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:30.042379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1d81678f-9032-4d70-ae85-f81ad6b20228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:30.043879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1025/2020] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:29 2022] DELETE /identity/v3/projects/21878b013b8a49d7a0aebfd7705faefe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:30.048761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.049230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.049338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.068303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:30.069561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:30.142440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 996/2021] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:30.150225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.150351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.151113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.171482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:30.172674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:30.183824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1026/2022] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:30.188598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.199675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b6512-253a-4585-aa11-7fa1677348cd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.205148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.205978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.206157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.206533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.207279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:30.207673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.209390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.222103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 997/2023] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:25:30.226630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.236685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406c84-6d2d-4f7f-bd64-379841f7e48a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.241490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.242091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.242369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.242673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.243319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.246192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.261374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:30.279985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1027/2024] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:30 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:30.283654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.302763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf373e37-6bc5-402a-a429-98339fe0a87d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.350492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.351043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.351323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.351595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.352229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.354873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.363890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:30.377470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 998/2025] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:30 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:30.381632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.392517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e7ccf-fe3e-4079-ae72-e8b68d280bbe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.397538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.398133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.398406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.398710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.399346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.401014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.413268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1028/2026] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:30.414624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.424785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d94a03-9281-4257-b821-2f178979c027 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.429410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.429987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.430263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.430532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.431200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.433379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3ef48-d073-40e8-990a-661e8e89bfe7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.446658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 999/2027] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:30.451096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.463578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7dd1b-c81d-486b-9236-cd0446c596d9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.470465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.471161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.471440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.471709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] PATH_INFO: `/v3/projects/2af8940da97b4298a92e1d45f70cfe29/users/fd862cc69267470580ef23e0223742e0/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.487394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af8940da97b4298a92e1d45f70cfe29, user_id=fd862cc69267470580ef23e0223742e0, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.490094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.505508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1029/2028] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:30 2022] PUT /identity/v3/projects/2af8940da97b4298a92e1d45f70cfe29/users/fd862cc69267470580ef23e0223742e0/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:30.510978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.511263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.511592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.530858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:30.532060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:30.591101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1000/2029] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:30.600684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:30.602358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:30.668574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3336251-4074-4ace-b247-95bf99387e9f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:30.672236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:30.672851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:30.673127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:30.673400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:30.683671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:30.685205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba7fff-7df4-4071-88ae-f9555fb2b22f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:30.699965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1030/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:35.895271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:35.896647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:35.922926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920a6f1-9d78-4207-a167-e29acb91f153 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:35.938886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:35.939614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:35.940078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:35.940347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] PATH_INFO: `/v3/users/de11bdf931b245a9a8b519808fdc1d3c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:35.941614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de11bdf931b245a9a8b519808fdc1d3c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:35.943630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.013948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de11bdf931b245a9a8b519808fdc1d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:36.026078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de11bdf931b245a9a8b519808fdc1d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:36.026837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de11bdf931b245a9a8b519808fdc1d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:36.027888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de11bdf931b245a9a8b519808fdc1d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:36.036667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87997db6-3f39-4c2a-974a-02170254c383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de11bdf931b245a9a8b519808fdc1d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:36.037297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.037789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.037908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.061014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1001/2031] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:35 2022] DELETE /identity/v3/users/de11bdf931b245a9a8b519808fdc1d3c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:36.062440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.062863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.063438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.074015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] MFA Rules not processed for user `fd862cc69267470580ef23e0223742e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:36.075415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:36.103247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:36.105104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:36.167301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1031/2032] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:36.172326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:36.174192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.176923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:36.192351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1002/2033] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:36.210020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a682c1-f130-4e63-b994-bb0f1cca14fb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.216338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.218854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:36.232503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.233501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.233953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.234390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.235266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.237400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.298265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1032/2034] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:36 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:36.301503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.347975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:36.357650 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108559a-65b4-4bce-9932-0eb313ea0957 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.435105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1b960-3ddb-4325-8e4a-0c1f54b9d071 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.476512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.480302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.480587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.480860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.484481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.490622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.505755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.506463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.506773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.507049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.541313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.546617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.559996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1003/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:36.571852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1033/2036] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:36 2022] POST /identity/v3/users => generated 299 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:36.578537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.596756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c23c4-7ceb-4919-881a-7997510be39c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.602739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.603531 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.603812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.604083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.605096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.607590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.627321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1004/2037] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:36.642173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.665454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24636879-c7ea-4571-b4a0-be3e5288f787 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.671841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.672903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.672903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.673093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] PATH_INFO: `/v3/projects/203da2a4b1de47bdae454841113a2fa9/users/94b8657a6b6042dca12872ed16854596/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.692450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=203da2a4b1de47bdae454841113a2fa9, user_id=94b8657a6b6042dca12872ed16854596, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.697753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.720905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1034/2038] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:36 2022] PUT /identity/v3/projects/203da2a4b1de47bdae454841113a2fa9/users/94b8657a6b6042dca12872ed16854596/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:36.726645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.741869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4394b-67ab-4989-9469-18e8fd901963 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.816362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.817283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.817767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.818181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.819134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.821098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.832990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1005/2039] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:36.837136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.850337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af490ec5-ab2d-47c0-8313-aaf558c7e9fc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:36.856840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:36.858169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:36.858468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:36.858790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] PATH_INFO: `/v3/projects/203da2a4b1de47bdae454841113a2fa9/users/94b8657a6b6042dca12872ed16854596/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:36.860950 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=203da2a4b1de47bdae454841113a2fa9, user_id=94b8657a6b6042dca12872ed16854596, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:36.863835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:36.874947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1035/2040] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:36 2022] PUT /identity/v3/projects/203da2a4b1de47bdae454841113a2fa9/users/94b8657a6b6042dca12872ed16854596/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:36.922226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:36.924353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:36.939646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb700e-2d49-4c71-b427-f444c74135da admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:37.004030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:37.042931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:37.043739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:37.044025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:37.044294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] PATH_INFO: `/v3/projects/c4f0ac1be49c4e5dafb42d2a35ae45fa` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:37.046089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f0ac1be49c4e5dafb42d2a35ae45fa)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:37.048208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:37.112028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f0ac1be49c4e5dafb42d2a35ae45fa', 'request_id': 'req-3b5598c4-0d7f-4ad3-8d38-61d969314cde'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.121368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f0ac1be49c4e5dafb42d2a35ae45fa', 'request_id': 'req-3b5598c4-0d7f-4ad3-8d38-61d969314cde'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.135698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f0ac1be49c4e5dafb42d2a35ae45fa', 'request_id': 'req-3b5598c4-0d7f-4ad3-8d38-61d969314cde'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.148543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] The token cache is being invalidate because project c4f0ac1be49c4e5dafb42d2a35ae45fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:37.148889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.149567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.151250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1006/2041] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:36 2022] DELETE /identity/v3/projects/c4f0ac1be49c4e5dafb42d2a35ae45fa => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:37.155384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:37.157646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:37.174697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed644cfb-a2cd-4445-be0c-d09b48b3ef4f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:37.245790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:37.246439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:37.246754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:37.247028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] PATH_INFO: `/v3/users/206e440512ce4a8d88ccf1454a32eee9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:37.248163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=206e440512ce4a8d88ccf1454a32eee9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:37.250308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:37.316934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206e440512ce4a8d88ccf1454a32eee9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.326794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206e440512ce4a8d88ccf1454a32eee9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.329740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206e440512ce4a8d88ccf1454a32eee9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.331136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206e440512ce4a8d88ccf1454a32eee9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.352695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5c4a31e3-4325-4d7b-80a6-c1ebf79fcd7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206e440512ce4a8d88ccf1454a32eee9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:37.370305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:37.378477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1036/2042] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:37 2022] DELETE /identity/v3/users/206e440512ce4a8d88ccf1454a32eee9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:37.778750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:37.794349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5598c4-0d7f-4ad3-8d38-61d969314cde admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:37.941635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:37.943153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:37.943575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:37.944158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] PATH_INFO: `/v3/projects/0955fcc17e704b17939c0a8c0100e425` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:37.946375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0955fcc17e704b17939c0a8c0100e425)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:37.948815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.059014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0955fcc17e704b17939c0a8c0100e425', 'request_id': 'req-5453bb78-0008-46c9-8dd5-5997303c890f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:38.072775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0955fcc17e704b17939c0a8c0100e425', 'request_id': 'req-5453bb78-0008-46c9-8dd5-5997303c890f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:38.087222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0955fcc17e704b17939c0a8c0100e425', 'request_id': 'req-5453bb78-0008-46c9-8dd5-5997303c890f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:38.112556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:38.127055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] The token cache is being invalidate because project 0955fcc17e704b17939c0a8c0100e425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:38.127314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:38.132570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5453bb78-0008-46c9-8dd5-5997303c890f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:38.137601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1007/2043] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:37 2022] DELETE /identity/v3/projects/0955fcc17e704b17939c0a8c0100e425 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:38.143032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.143657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.144173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.185347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:38.188471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:38.285527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1037/2044] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:38.295342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.295721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.296031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.320789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:38.322409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:38.333735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1008/2045] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:38.338605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.351264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4949bd-15eb-4aab-a66e-cc99325ee1d3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.357604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.359440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.360294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.360936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.363280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:38.364163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.366814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.382990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1038/2046] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:38.387173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.399422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a7bd6-ae6a-4cf3-9aee-f1689cc5e045 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.405579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.406320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.406717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.407062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.407756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.410048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.446733 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1009/2047] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:38.447695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.463219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8f207-1c39-4623-bebd-3a56a4e58a8b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.539501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.540114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.540389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.540818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.542006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.544106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.573416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1039/2048] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:38.576903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.589484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c73d7-9bf5-454f-a163-56d5a8fced8e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.594998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.595395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.595588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.595862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.596561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.600458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.615395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1010/2049] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:38.616319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.629619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac003a3-abee-4cb3-a793-8e2c7a2e6e15 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.635538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.636110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.636416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.636686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.637749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.639114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68e51e1-eda7-4240-8077-ce9f7a62b59d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.654881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1040/2050] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:25:38.657101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.671197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefaf71-d44b-42e4-bb9a-f29a962bf98a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.679098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.680011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.680501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.680947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] PATH_INFO: `/v3/projects/8e3ed5696818453aa3072f4eef6ff37b/users/0bfc322e39cc4f70bad412a4c53526e0/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.699835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3ed5696818453aa3072f4eef6ff37b, user_id=0bfc322e39cc4f70bad412a4c53526e0, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.703430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.721308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1011/2051] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:38 2022] PUT /identity/v3/projects/8e3ed5696818453aa3072f4eef6ff37b/users/0bfc322e39cc4f70bad412a4c53526e0/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:38.732885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.734064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.734570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.769874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] MFA Rules not processed for user `0bfc322e39cc4f70bad412a4c53526e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:38.771545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:38.836364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:38.848325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1041/2052] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:38.852376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.930189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3bea6-96bf-4f82-9aac-3db913abc086 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:38.936081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:38.936888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:38.937742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:38.938201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:38.939056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:38.941225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:38.969981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1012/2053] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:38.974096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:38.990043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896708c-2e0c-4297-85b6-2dd8a80785fb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.064594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.065217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.066858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.067379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.068475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.070803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.128691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1042/2054] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:38 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:39.129608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.141202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9789f1-b9b2-4126-994a-8b03c1c1ffc3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.161431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.167560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.171001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.173506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.174566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.181961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.214273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1013/2055] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:39.218423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.240082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895faf9-b638-40ae-8202-1d70f2106797 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.255569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.258827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.260108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.276309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.276309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.276309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abaef7-d48d-4ede-bbce-2dead970f89d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.294078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1043/2056] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:39.300107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.315360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf383a-a800-4851-9349-99f31b7a7393 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.330051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.330761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.331042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.331316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] PATH_INFO: `/v3/projects/b7605d4e907545ce9493b0757218b5d1/users/82cb9d9c861740dba1ce50aa389f8d66/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.368731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7605d4e907545ce9493b0757218b5d1, user_id=82cb9d9c861740dba1ce50aa389f8d66, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.372350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.393482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1014/2057] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:39 2022] PUT /identity/v3/projects/b7605d4e907545ce9493b0757218b5d1/users/82cb9d9c861740dba1ce50aa389f8d66/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:39.404049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.404338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.404799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.473280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] MFA Rules not processed for user `82cb9d9c861740dba1ce50aa389f8d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:39.474551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:39.604228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1044/2058] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:39.608379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.683547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:39.725604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745511e5-af01-4d51-8d60-384953615387 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.731909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.732538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.732823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.733097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.733739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.736023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.780685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1015/2059] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:39 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:39.784519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.799505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a25d9-b6ed-42d9-a970-ad2647929b37 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:39.884028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:39.884758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:39.885445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:39.886092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:39.886800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:39.889034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:39.901108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:39.964141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:39.964903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1045/2060] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:39 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:40.015939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf399c-c1a7-4bdb-9a68-5d9977a186ee admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:40.027292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:40.027903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.028181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.028454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.029109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:40.031387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:40.055536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:40.068080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c28a8-0598-4458-8410-a331d7f8bd47 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:40.069088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1016/2061] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:40.074366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:40.075141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.076057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.076345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] PATH_INFO: `/v3/projects/0bfba5ed73f2483bb15678711f17b9eb/users/62a25711263d413b8c89698a6857b45e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.096938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfba5ed73f2483bb15678711f17b9eb, user_id=62a25711263d413b8c89698a6857b45e, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:40.099887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:40.140568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:40.143819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1046/2062] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:40 2022] PUT /identity/v3/projects/0bfba5ed73f2483bb15678711f17b9eb/users/62a25711263d413b8c89698a6857b45e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:40.208550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c68b7-9dd0-4cb9-93cb-e5b9e8267288 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:40.444574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:40.446780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:40.550258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:40.551065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.551177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.551736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.552532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:40.554597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:40.602695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1017/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:40.605262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:40.623679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c29b-89c8-43e5-be32-c484ba200418 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:40.626599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53b71-c114-469b-9f09-fc674d1f6af6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:40.634711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:40.635365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.635644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.635911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] PATH_INFO: `/v3/users/ed8c2fc2f2bb44f1b172019b0e0158e2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.637023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:40.637684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.637962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.638227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] PATH_INFO: `/v3/projects/0bfba5ed73f2483bb15678711f17b9eb/users/62a25711263d413b8c89698a6857b45e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.640811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8c2fc2f2bb44f1b172019b0e0158e2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:40.642232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:40.644585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfba5ed73f2483bb15678711f17b9eb, user_id=62a25711263d413b8c89698a6857b45e, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:40.647271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaaa30-8454-4b47-ba28-764d6dd73766 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:40.673702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1018/2064] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:40 2022] PUT /identity/v3/projects/0bfba5ed73f2483bb15678711f17b9eb/users/62a25711263d413b8c89698a6857b45e/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:40.685700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.685984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.686256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.768945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8c2fc2f2bb44f1b172019b0e0158e2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:40.776859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] MFA Rules not processed for user `62a25711263d413b8c89698a6857b45e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:40.783709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:40.783906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8c2fc2f2bb44f1b172019b0e0158e2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:40.784804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8c2fc2f2bb44f1b172019b0e0158e2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:40.785432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8c2fc2f2bb44f1b172019b0e0158e2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:40.808390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-14948da9-d684-4057-be4b-29dd98b19c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8c2fc2f2bb44f1b172019b0e0158e2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:40.857883 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1047/2065] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:40 2022] DELETE /identity/v3/users/ed8c2fc2f2bb44f1b172019b0e0158e2 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:40.863783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:40.864082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:40.864332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:40.914479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:40.916306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:40.954141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1019/2066] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:40.964867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:40.983109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:41.071952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:41.100680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:41.114092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1048/2067] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:41.178500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cb9dd-36ea-446c-88d6-f913750e850a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:41.201395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:41.203024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:41.343455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:41.345300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:41.345872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:41.346468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:41.446397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f733dc8-88b2-444a-94a0-753169ff3bb6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:41.491580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:41.507280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:41.508060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:41.508831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:41.553847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:41.555787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:41.596277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1049/2068] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:41.680690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:41.682375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:41.723510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1020/2069] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:41.794943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:41.816238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4297a47-e86d-49f5-a28a-9bee7ac534fa service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:41.829836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:41.830392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:41.830702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:41.830977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:41.832831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:42.091229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:42.092753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296cfa0-8269-4fd0-92bc-4e3cc2247372 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:42.123117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1050/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:43.432756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:43.434386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:43.471717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dcd0c-4773-4c41-b86e-b78698d3ea97 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:43.484095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:43.485334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:43.485334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:43.485334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] PATH_INFO: `/v3/projects/242344b2aa9d445d8504956b1fc5655c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:43.486965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242344b2aa9d445d8504956b1fc5655c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:43.488442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:43.561183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242344b2aa9d445d8504956b1fc5655c', 'request_id': 'req-868ad9b5-323e-4194-b219-98e19f425f78'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:43.569696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242344b2aa9d445d8504956b1fc5655c', 'request_id': 'req-868ad9b5-323e-4194-b219-98e19f425f78'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:43.597067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242344b2aa9d445d8504956b1fc5655c', 'request_id': 'req-868ad9b5-323e-4194-b219-98e19f425f78'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:43.625622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] The token cache is being invalidate because project 242344b2aa9d445d8504956b1fc5655c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:43.625941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:43.627049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-868ad9b5-323e-4194-b219-98e19f425f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:43.632768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1021/2071] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:43 2022] DELETE /identity/v3/projects/242344b2aa9d445d8504956b1fc5655c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:43.637528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:43.637995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:43.638090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:43.676722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:43.678000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:43.751779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1051/2072] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:43.755335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:43.755782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:43.756539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:43.793646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:43.795064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:43.805036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1022/2073] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:43.809580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:43.824137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fb0b9-a4f4-4385-b327-fcf86e97e3ab None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:43.840067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:43.840987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:43.841410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:43.841822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:43.842663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:43.843120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:43.844889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:43.857586 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1052/2074] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:43.862042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:43.876886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff724c4-aa45-4f28-8ede-32fb64e25931 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:43.893441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:43.894097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:43.894382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:43.894877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:43.895540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:43.898124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:43.906792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:43.953564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1023/2075] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:43 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:43.954840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:43.967122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fef2b-7be6-450c-b297-436ae05c0cc9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:43.996620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:44.099334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.099972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.100251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.101039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.101685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.106138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.179911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1053/2076] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:43 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:44.185550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:44.211279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b78068-b666-47be-942d-750c3f99fe08 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:44.225474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.227109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.227403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.228785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.229460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.233579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.260966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1024/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:44.264570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:44.281929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f417-948f-4a26-baab-582ded00a541 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:44.295608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.298935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.298935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.298935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.299983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.306654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b84013-fdd8-4ab1-be8d-3bcf742707dd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.329638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1054/2078] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:44.333857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:44.349036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cac4c-f604-4214-829f-0129107b7952 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:44.366849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.367764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.370334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.372421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] PATH_INFO: `/v3/projects/89b44de54291465a8b30343a11d72f78/users/4d1f91620b8e4d87b7ce4a015b0d447d/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.393401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b44de54291465a8b30343a11d72f78, user_id=4d1f91620b8e4d87b7ce4a015b0d447d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.397148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f9bad-d671-4d18-9f89-507f4b5ff768 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.414055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.414640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.415013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.442494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1025/2079] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:44 2022] PUT /identity/v3/projects/89b44de54291465a8b30343a11d72f78/users/4d1f91620b8e4d87b7ce4a015b0d447d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:44.454653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.454653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.454653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.486005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] MFA Rules not processed for user `94b8657a6b6042dca12872ed16854596`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:44.487769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:44.506355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] MFA Rules not processed for user `4d1f91620b8e4d87b7ce4a015b0d447d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:44.508540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:44.610740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1026/2080] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:44.624515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:44.626255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:44.656876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1055/2081] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:44.671302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:44.673071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:44.720204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3923492-5c69-4a8b-bb4f-fb673b11b3e7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:44.725073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.727003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.727003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.727003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.745009 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.751515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369409cf-cb04-4220-9f2f-da88319ae9d2 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.765220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185851e8-384f-4cc7-b7d5-b0346804f6f6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:44.769833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:44.770800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:44.771748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:44.772223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:44.774543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:44.783000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1027/2082] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:44.826191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:44.876895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:44.878874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:44.908350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1056/2083] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:45.109082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472c62-52aa-4807-872f-eb38698d454c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:45.109366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472c62-52aa-4807-872f-eb38698d454c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:45.109642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5472c62-52aa-4807-872f-eb38698d454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:45.142402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a5472c62-52aa-4807-872f-eb38698d454c None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:45.143936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a5472c62-52aa-4807-872f-eb38698d454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:45.202192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:45.262333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1028/2084] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:25:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:45.320019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:45.322674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:45.483648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3102ae2-7d7a-42ec-9a1e-c90356033bcf service glance] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:45.493188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:45.493907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:45.494185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:45.494629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:45.509018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:45.511486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3abd37-238a-47c0-a544-169bf0968862 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:45.529201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1057/2085] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:25:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:25:46.167810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:46.168245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:46.168408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:46.172180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1029/2086] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:25:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:25:46.189252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:46.189806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:46.189906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:46.231652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:46.235742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:46.280409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:46.286275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:46.421553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1058/2087] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:25:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:46.427827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:46.429784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:46.452745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb641d8-dc99-455b-af03-327df0cd942b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:46.457243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a551bc-3ac3-4c94-be1a-0871335a8a4d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:46.467721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:46.469122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:46.470304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:46.470840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] PATH_INFO: `/v3/users/93d23721dfd34020b659707b65eb2c33` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:46.472249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:46.473124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:46.473602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:46.474088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:46.491877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d23721dfd34020b659707b65eb2c33)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:46.498693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:46.527316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:46.529428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:46.571511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1059/2088] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:46.620439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d23721dfd34020b659707b65eb2c33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:46.630386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d23721dfd34020b659707b65eb2c33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:46.631386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d23721dfd34020b659707b65eb2c33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:46.632029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d23721dfd34020b659707b65eb2c33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:46.645176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0326159f-4fcf-4324-adfb-e6c8454cd941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d23721dfd34020b659707b65eb2c33'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:46.670829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1030/2089] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:46 2022] DELETE /identity/v3/users/93d23721dfd34020b659707b65eb2c33 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:46.927628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:46.948805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5392ba06-17ad-44b0-89a2-743aef7c1acc service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.084580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:47.100053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.100942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.101384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.101804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] PATH_INFO: `/v3/projects/22ba57c2ca3e4ff9a4b77d8cecbfa4e9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.103747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ba57c2ca3e4ff9a4b77d8cecbfa4e9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.106828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:47.245497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'request_id': 'req-5db45a3f-de49-43c1-8830-ca20922bb956'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:47.253008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'request_id': 'req-5db45a3f-de49-43c1-8830-ca20922bb956'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:47.266967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ba57c2ca3e4ff9a4b77d8cecbfa4e9', 'request_id': 'req-5db45a3f-de49-43c1-8830-ca20922bb956'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:47.281122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] The token cache is being invalidate because project 22ba57c2ca3e4ff9a4b77d8cecbfa4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:47.281515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:47.282616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5db45a3f-de49-43c1-8830-ca20922bb956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:47.290641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1060/2090] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:46 2022] DELETE /identity/v3/projects/22ba57c2ca3e4ff9a4b77d8cecbfa4e9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:47.293368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.294374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.294880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.319030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:47.320616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:47.394638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1031/2091] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:47.402936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.403375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.403786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.426548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:47.427960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:47.444989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:47.470848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1061/2092] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:47.493112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2811beb-8e6f-41b2-ad1a-db03ec660818 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.503222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.504140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.504784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.505253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.506209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:47.506773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.510192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f557ca-adae-4d45-b048-e225329132f5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:47.514495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:47.536173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1032/2093] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:47.536749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:47.565475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea90a1-c2da-49e9-b857-3a490864d0cf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.579316 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.579962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.580251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.580946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.581671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.589119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:47.642448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.643592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.644584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.668266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1062/2094] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:47 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:47.669147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:47.693634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] MFA Rules not processed for user `4d1f91620b8e4d87b7ce4a015b0d447d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:47.695228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:47.798133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1033/2095] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:47.820760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e058f-8003-4983-839e-1d1718b61f67 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.832366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.833796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.833796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.833796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.835136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.839526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:47.902285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1063/2096] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:47 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:47.906247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:47.927841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5c96-8fa0-4136-9e02-d292c38c204b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.933232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.934031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.934340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.934679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.935581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.937920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:47.956803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1034/2097] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:47.957636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:47.969290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca6164-a3a7-425f-8203-afa8fec1535b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:47.977613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:47.978936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:47.979255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:47.979640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:47.980358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:47.982349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d092-e644-48f1-8c30-b2e3bacaa269 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:48.002627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1064/2098] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:48.004115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:48.017496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeca83-f120-47d8-b2ea-1c87f9dd8aa2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:48.023960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:48.024563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:48.025386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:48.025664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] PATH_INFO: `/v3/projects/6fbf7eeb158e4a72bd8133faa1366e0d/users/00399347c76d451d94d45f4edc3cf416/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:48.080137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fbf7eeb158e4a72bd8133faa1366e0d, user_id=00399347c76d451d94d45f4edc3cf416, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:48.085987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:48.127606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1035/2099] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:48 2022] PUT /identity/v3/projects/6fbf7eeb158e4a72bd8133faa1366e0d/users/00399347c76d451d94d45f4edc3cf416/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:48.134093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:48.134424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:48.134735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:48.177064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:48.180758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:48.282865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:48.374965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1065/2100] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:48.388502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:48.390763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:48.546947 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f447b5c-1908-43e0-bb9c-b3a010f135c2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:48.557692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:48.558318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:48.560332 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:48.560584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:48.582303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:48.584010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca9ee1-9acc-4049-9136-abe3a9d4a437 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:48.629078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1036/2101] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:49.598587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:49.602461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:49.615595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051f9ad-ea9a-4553-97f4-3cef2e6421f7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:49.625207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:49.625852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:49.626130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:49.626400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] PATH_INFO: `/v3/users/0bfc322e39cc4f70bad412a4c53526e0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:49.643131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfc322e39cc4f70bad412a4c53526e0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:49.645104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:49.772613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfc322e39cc4f70bad412a4c53526e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:49.783622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfc322e39cc4f70bad412a4c53526e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:49.785342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfc322e39cc4f70bad412a4c53526e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:49.786240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfc322e39cc4f70bad412a4c53526e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:49.804382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfc322e39cc4f70bad412a4c53526e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:49.835479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1066/2102] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:49 2022] DELETE /identity/v3/users/0bfc322e39cc4f70bad412a4c53526e0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:49.844367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:49.844853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:49.844930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:49.897449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:49.903084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:50.092015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1037/2103] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:50.104105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:50.105968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:50.467097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9efda-6787-4a39-ad68-aa923e8b7344 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:50.476977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:50.483433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:50.487054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:50.489254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:50.518387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:50.525949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:50.542890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1067/2104] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:51.346590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:51.349825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:51.375498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90ff28-0b5a-44f5-96fe-ca75d82f13f0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:51.386827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:51.387860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:51.388362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:51.388854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] PATH_INFO: `/v3/projects/8e3ed5696818453aa3072f4eef6ff37b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:51.390404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3ed5696818453aa3072f4eef6ff37b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:51.392871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:51.395900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:51.460546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3ed5696818453aa3072f4eef6ff37b', 'request_id': 'req-14eb4a30-4c87-4d58-af16-695dad0f9d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.479255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3ed5696818453aa3072f4eef6ff37b', 'request_id': 'req-14eb4a30-4c87-4d58-af16-695dad0f9d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.503824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3ed5696818453aa3072f4eef6ff37b', 'request_id': 'req-14eb4a30-4c87-4d58-af16-695dad0f9d3c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.531151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] The token cache is being invalidate because project 8e3ed5696818453aa3072f4eef6ff37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:51.532667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.533674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.536950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1038/2105] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:51 2022] DELETE /identity/v3/projects/8e3ed5696818453aa3072f4eef6ff37b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:51.541394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:51.553751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e600e4-515d-40c5-8c24-55028bcee829 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:51.675513 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:51.721809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:51.722456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:51.722769 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:51.723049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] PATH_INFO: `/v3/users/82cb9d9c861740dba1ce50aa389f8d66` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:51.727486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82cb9d9c861740dba1ce50aa389f8d66)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:51.736982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:51.839502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82cb9d9c861740dba1ce50aa389f8d66'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.853713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82cb9d9c861740dba1ce50aa389f8d66'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.854363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82cb9d9c861740dba1ce50aa389f8d66'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.855300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82cb9d9c861740dba1ce50aa389f8d66'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.881760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82cb9d9c861740dba1ce50aa389f8d66'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:51.908646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1068/2106] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:51 2022] DELETE /identity/v3/users/82cb9d9c861740dba1ce50aa389f8d66 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:52.700431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:52.701928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:52.719542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb4a30-4c87-4d58-af16-695dad0f9d3c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:52.828183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:52.829165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:52.829583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:52.830003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] PATH_INFO: `/v3/projects/b7605d4e907545ce9493b0757218b5d1` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:52.831414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7605d4e907545ce9493b0757218b5d1)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:52.833466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:52.904604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7605d4e907545ce9493b0757218b5d1', 'request_id': 'req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:52.913137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7605d4e907545ce9493b0757218b5d1', 'request_id': 'req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:52.926538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7605d4e907545ce9493b0757218b5d1', 'request_id': 'req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:52.946496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] The token cache is being invalidate because project b7605d4e907545ce9493b0757218b5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:52.946852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:52.947619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:52.949741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1039/2107] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:52 2022] DELETE /identity/v3/projects/b7605d4e907545ce9493b0757218b5d1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:52.953533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:52.955039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:52.972271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405365e8-565f-4440-8a77-5eb86cd3bfa3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:53.069446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:53.070627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:53.071736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:53.072601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] PATH_INFO: `/v3/users/62a25711263d413b8c89698a6857b45e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:53.081205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a25711263d413b8c89698a6857b45e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:53.084543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:53.192983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a25711263d413b8c89698a6857b45e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:53.208632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a25711263d413b8c89698a6857b45e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:53.209362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a25711263d413b8c89698a6857b45e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:53.210112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a25711263d413b8c89698a6857b45e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:53.222332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:53.246479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8e4a8f68-64c0-48dc-a159-ccd1c4f0c31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a25711263d413b8c89698a6857b45e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:53.294715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1069/2108] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:52 2022] DELETE /identity/v3/users/62a25711263d413b8c89698a6857b45e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:54.115896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:54.119451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:54.151882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b58eb5-b9b9-43e3-906d-01cc3b131dec admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:54.198709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:54.323244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:54.324365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:54.324881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:54.325377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] PATH_INFO: `/v3/projects/0bfba5ed73f2483bb15678711f17b9eb` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:54.327242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfba5ed73f2483bb15678711f17b9eb)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:54.334713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:54.508069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfba5ed73f2483bb15678711f17b9eb', 'request_id': 'req-91cd88e4-6bad-4c6f-bedb-5f83547249bc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:54.528525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfba5ed73f2483bb15678711f17b9eb', 'request_id': 'req-91cd88e4-6bad-4c6f-bedb-5f83547249bc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:54.548337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfba5ed73f2483bb15678711f17b9eb', 'request_id': 'req-91cd88e4-6bad-4c6f-bedb-5f83547249bc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:54.575972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] The token cache is being invalidate because project 0bfba5ed73f2483bb15678711f17b9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:25:54.576479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:54.577532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91cd88e4-6bad-4c6f-bedb-5f83547249bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:54.581179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1040/2109] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:25:54 2022] DELETE /identity/v3/projects/0bfba5ed73f2483bb15678711f17b9eb => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:54.585723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:54.586236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:54.586362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:54.655382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:54.661005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:54.836353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1070/2110] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:54.844616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:54.845252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:54.845804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:54.891601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:54.894553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:54.917894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1041/2111] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:25:54.922166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:54.938090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247446a4-9a2c-4041-90bd-5f1cb487e5e1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:54.947726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:54.948437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:54.948804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:54.949143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:54.949872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:25:54.950241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:54.952436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:54.965725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1071/2112] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:25:54.970460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:54.984027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d0bf5-d821-47e1-90cf-2d176c667bbb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:54.991451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:54.992210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:54.993111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:54.993834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:54.994906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:54.997941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.078041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1042/2113] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:54 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:25:55.078985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.096343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28539-6641-4c39-8b82-6e980851cf37 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:55.167811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:55.168988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.169391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.169996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.170606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:55.173557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.209161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1072/2114] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:55 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:55.213189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.228730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a51be-0897-4048-b3c3-00f508d10911 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:55.246744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:55.247597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.248029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.248440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.249554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:55.251329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.275621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1043/2115] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:55.281464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.296129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d04d6-3b9b-4b52-a412-9384011776b8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:55.307186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:55.308436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.310238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.310657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.312944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:55.320578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936963e7-87cc-44b2-8e79-b97c92121153 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.350344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1073/2116] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:55.351447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.377142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6a38-be94-4026-85c3-5ea9a8eeb055 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:55.383384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:55.384532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.385049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.385498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] PATH_INFO: `/v3/projects/e7ad10f72d784a6bb9321b43b22e8172/users/eeb1c7c3158b4014ad50c8069ebe80b4/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.412562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ad10f72d784a6bb9321b43b22e8172, user_id=eeb1c7c3158b4014ad50c8069ebe80b4, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:55.415975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.434379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1044/2117] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:55 2022] PUT /identity/v3/projects/e7ad10f72d784a6bb9321b43b22e8172/users/eeb1c7c3158b4014ad50c8069ebe80b4/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:55.448155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.448444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.448713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.504816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] MFA Rules not processed for user `eeb1c7c3158b4014ad50c8069ebe80b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:55.506356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:55.554933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:55.615237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.621966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1074/2118] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:55.756213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:55.815308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706e2c5-ffc2-43c5-a61d-67fe10193bdf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:55.834526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:55.835558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:55.836079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:55.836606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:55.837998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:55.840180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:55.927360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1045/2119] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:25:55 2022] POST /identity/v3/projects => generated 381 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:55.927571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:55.954817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3af05-fe2b-4757-ae91-0b9c5c86ecd2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:56.076325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:56.076972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.077715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.077993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.078722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:56.081295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:56.123475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:56.131885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1075/2120] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:25:55 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:25:56.138786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938d253-ece8-495b-aa4f-6ba38e4ed15f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:56.166513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:56.167772 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.168219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.168756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.169686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:56.171407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:56.189618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1046/2121] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:56.190927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:56.218964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59211c1-2d58-4f3c-af7c-a6a136a19991 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:56.228765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:56.229426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.229691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.229964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] PATH_INFO: `/v3/projects/c1335fdc89e148cc888f7bd04462c724/users/2188e84b17324f9993f2a5afde42ad7d/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.266060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1335fdc89e148cc888f7bd04462c724, user_id=2188e84b17324f9993f2a5afde42ad7d, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:56.269866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:56.322624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1076/2122] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:56 2022] PUT /identity/v3/projects/c1335fdc89e148cc888f7bd04462c724/users/2188e84b17324f9993f2a5afde42ad7d/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:25:56.326714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:56.338675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da02b1e-f3da-4afa-bc7b-f2b162c5b804 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:56.574663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:56.575408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.576769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.576769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.578623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:56.581831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65dd82e-25a0-4b2b-826c-f787d1460a52 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:56.611201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1047/2123] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:25:56.612011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:56.625264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c7ec5-3b83-463d-a800-2443020a04b9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:56.644932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:56.645628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.645909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.646185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] PATH_INFO: `/v3/projects/c1335fdc89e148cc888f7bd04462c724/users/2188e84b17324f9993f2a5afde42ad7d/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.648271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1335fdc89e148cc888f7bd04462c724, user_id=2188e84b17324f9993f2a5afde42ad7d, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:56.651132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:56.719129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1077/2124] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:25:56 2022] PUT /identity/v3/projects/c1335fdc89e148cc888f7bd04462c724/users/2188e84b17324f9993f2a5afde42ad7d/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:56.723303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658d551-6951-4362-bccc-ae8c7b3a6892 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:56.723576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658d551-6951-4362-bccc-ae8c7b3a6892 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:56.723859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658d551-6951-4362-bccc-ae8c7b3a6892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:56.831560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b658d551-6951-4362-bccc-ae8c7b3a6892 None None] MFA Rules not processed for user `2188e84b17324f9993f2a5afde42ad7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:56.832837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b658d551-6951-4362-bccc-ae8c7b3a6892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:57.156166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1048/2125] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:57.184798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:57.187308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:57.401111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dea7ab-7bc1-44a5-9d3b-b332cfb274c2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:57.427449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:57.428103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:57.429619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:57.429619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:57.443432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:57.444984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:57.472298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1078/2126] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:25:57.611323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:57.611811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:57.611907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:57.683794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] MFA Rules not processed for user `00399347c76d451d94d45f4edc3cf416`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:57.685411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:57.712096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:57.722964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3fc74-7005-46fa-ba7a-2bfde0eddc5c service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:57.735137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:57.736253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:57.736739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:57.737267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:57.768477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:57.804338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:57.844377 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1049/2127] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:57.853946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:57.855749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:57.865594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:57.867107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:57.886504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1079/2128] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:58.122998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd313e05-705b-4f2a-9701-afd67c8306ca None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:58.142627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:58.143265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:58.143543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:58.143816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:58.159333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:58.162765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df1e3f-0692-48ea-a595-5485907d37de service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:58.202823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1050/2129] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:25:58.499218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:58.808654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae6e6-2998-4c74-978d-9023241b2b7b service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:58.819747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:58.820788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:58.821265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:58.824005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] PATH_INFO: `/v3/users/eeb1c7c3158b4014ad50c8069ebe80b4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:58.825964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb1c7c3158b4014ad50c8069ebe80b4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:58.828562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:58.943329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb1c7c3158b4014ad50c8069ebe80b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:58.960894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb1c7c3158b4014ad50c8069ebe80b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:58.962350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb1c7c3158b4014ad50c8069ebe80b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:58.985724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb1c7c3158b4014ad50c8069ebe80b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:59.021291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb1c7c3158b4014ad50c8069ebe80b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:25:59.055590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1080/2130] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:25:58 2022] DELETE /identity/v3/users/eeb1c7c3158b4014ad50c8069ebe80b4 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:25:59.060795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:59.061367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:59.061367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:59.094091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:25:59.096797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:59.222323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1051/2131] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:25:59.506863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:25:59.509351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:25:59.768575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7bed-605d-4bff-9ade-ffba6c58ecac admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:25:59.778078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:25:59.781105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:25:59.781407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:25:59.781678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:25:59.796742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:25:59.808057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:25:59.809547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:25:59.835188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1081/2132] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:00.830811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:00.833140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:00.868504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c39d2ab-3439-46e5-9b6b-b2b6430e6a28 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:00.884964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:00.887449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:00.888308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:00.889046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:00.889748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:00.890429 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] PATH_INFO: `/v3/projects/e7ad10f72d784a6bb9321b43b22e8172` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:00.893036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ad10f72d784a6bb9321b43b22e8172)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:00.901758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:00.917615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7205-db5f-4cbd-a5f5-09595f7e44fa service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:00.928869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:00.930525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:00.930850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:00.931124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] PATH_INFO: `/v3/users/52c75904c57745ea8b2b85b56da94b41` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:00.936419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c75904c57745ea8b2b85b56da94b41)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:00.945078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:01.012892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ad10f72d784a6bb9321b43b22e8172', 'request_id': 'req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.030386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ad10f72d784a6bb9321b43b22e8172', 'request_id': 'req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.033998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:01.050846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ad10f72d784a6bb9321b43b22e8172', 'request_id': 'req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.056611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c75904c57745ea8b2b85b56da94b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.078674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c75904c57745ea8b2b85b56da94b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.081972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c75904c57745ea8b2b85b56da94b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.083559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c75904c57745ea8b2b85b56da94b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.090891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] The token cache is being invalidate because project e7ad10f72d784a6bb9321b43b22e8172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:01.092027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.094154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.103301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1052/2133] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:00 2022] DELETE /identity/v3/projects/e7ad10f72d784a6bb9321b43b22e8172 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:01.103301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:01.105887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:01.120471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-072b7d13-5086-4e75-9830-02df9c077e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c75904c57745ea8b2b85b56da94b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.148682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1082/2134] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:00 2022] DELETE /identity/v3/users/52c75904c57745ea8b2b85b56da94b41 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:01.154277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:01.156119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:01.157800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:01.220669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:01.221897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:01.281353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeff7b6-41a9-46a2-b2a0-c3ef8d81d963 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:01.391712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1083/2135] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:01.405544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:01.407051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:01.442393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:01.442957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:01.443300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:01.443488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] PATH_INFO: `/v3/users/2188e84b17324f9993f2a5afde42ad7d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:01.471423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2188e84b17324f9993f2a5afde42ad7d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:01.473401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:01.630064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebbb9b-3b0f-4be2-ba23-8780b2466933 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:01.647402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2188e84b17324f9993f2a5afde42ad7d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.651702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:01.652193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:01.652474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:01.652745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:01.675594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2188e84b17324f9993f2a5afde42ad7d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.676803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2188e84b17324f9993f2a5afde42ad7d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.677259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2188e84b17324f9993f2a5afde42ad7d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.691375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:01.697139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2188e84b17324f9993f2a5afde42ad7d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:01.698443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d59f03-1fe0-4eea-a30e-0b12c2b36431 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:01.733209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1084/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:26:01.745344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1053/2137] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:01 2022] DELETE /identity/v3/users/2188e84b17324f9993f2a5afde42ad7d => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:02.718498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:02.719077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:02.719250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:02.753808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] MFA Rules not processed for user `00399347c76d451d94d45f4edc3cf416`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:02.755117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:02.762170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:02.860059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1085/2138] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:03.559822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:03.562176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:03.580343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:03.685843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:03.687724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:03.701713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12163da-be41-47e1-b2cf-2a9f8f582596 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:03.712042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:03.712956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:03.713206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:03.713553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] PATH_INFO: `/v3/projects/c1335fdc89e148cc888f7bd04462c724` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:03.714941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1335fdc89e148cc888f7bd04462c724)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:03.718388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:03.723206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff91e3f-6c33-4003-9d76-79f25c2df335 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:03.728554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:03.729314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:03.729828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:03.730203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] PATH_INFO: `/v3/projects/de5d17d6e9cf42edb66c9f66fc1e360d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:03.731641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5d17d6e9cf42edb66c9f66fc1e360d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:03.733826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:03.822807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5d17d6e9cf42edb66c9f66fc1e360d', 'request_id': 'req-1ae89796-502b-453b-a188-a7e0aeeee6c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.836001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5d17d6e9cf42edb66c9f66fc1e360d', 'request_id': 'req-1ae89796-502b-453b-a188-a7e0aeeee6c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.854687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1335fdc89e148cc888f7bd04462c724', 'request_id': 'req-91401fc4-7389-4372-8e58-dcbdf1b4d866'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.869843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5d17d6e9cf42edb66c9f66fc1e360d', 'request_id': 'req-1ae89796-502b-453b-a188-a7e0aeeee6c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.881533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1335fdc89e148cc888f7bd04462c724', 'request_id': 'req-91401fc4-7389-4372-8e58-dcbdf1b4d866'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.893678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] The token cache is being invalidate because project de5d17d6e9cf42edb66c9f66fc1e360d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:03.893983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.895065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1ae89796-502b-453b-a188-a7e0aeeee6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.900238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1086/2139] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:03 2022] DELETE /identity/v3/projects/de5d17d6e9cf42edb66c9f66fc1e360d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:03.903541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b636-69c4-491c-9dff-fc7b6d2b5218 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:03.903831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b636-69c4-491c-9dff-fc7b6d2b5218 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:03.904097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65b636-69c4-491c-9dff-fc7b6d2b5218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:03.911105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1335fdc89e148cc888f7bd04462c724', 'request_id': 'req-91401fc4-7389-4372-8e58-dcbdf1b4d866'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.929424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9f65b636-69c4-491c-9dff-fc7b6d2b5218 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:03.931952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9f65b636-69c4-491c-9dff-fc7b6d2b5218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:03.942607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] The token cache is being invalidate because project c1335fdc89e148cc888f7bd04462c724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:03.943166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.944903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-91401fc4-7389-4372-8e58-dcbdf1b4d866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:03.953539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1054/2140] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:03 2022] DELETE /identity/v3/projects/c1335fdc89e148cc888f7bd04462c724 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:03.967624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc714f-02a1-49ac-8199-b26853718d5f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:03.968435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc714f-02a1-49ac-8199-b26853718d5f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:03.968781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc714f-02a1-49ac-8199-b26853718d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:03.995972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-63cc714f-02a1-49ac-8199-b26853718d5f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:03.998038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-63cc714f-02a1-49ac-8199-b26853718d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:04.043231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1087/2141] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:04.045050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.045509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.045937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.073985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:04.075625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:04.089056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1088/2142] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:04.103847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.107472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1055/2143] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:04.115857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.115857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.116182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.134270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99521a4-2161-4123-85a1-5642da16cd18 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.142188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:04.143563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:04.153479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.154109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.154392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.154696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.155356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:26:04.155703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.156046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1056/2144] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:04.156976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.157281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.168925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4eefb2-f50c-46ab-91f3-846f30edc486 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.175673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1089/2145] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:26:04.182841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.184106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.184387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.185393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.187076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:26:04.187389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.189692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.190569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.204102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52380083-7f6a-4257-bf67-f650d4f426ea admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.212076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.212915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.213287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.213565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.214806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.217330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.230219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1057/2146] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:26:04.236057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.253977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0effb0a-afc8-4b5b-8a74-324390c59c02 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.260438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.261173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.261589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.261867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.262777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.265484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.267249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:04.292147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1090/2147] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:04.295020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:04.302075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.307330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1058/2148] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:26:04.315659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.406209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f21195-5dcb-4d15-8d4b-2a9da70db124 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.463469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f9dd2-b542-43ae-949a-c145be134dc6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.478123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.479176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.479666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.480159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.481021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.484089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.499077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.499757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.500040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.500311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.501059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.503225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.554067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1091/2149] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:04.558202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.560232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1059/2150] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:04.564071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.574536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f52e23-734b-4191-9871-c990ef7346aa admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.578528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc929b0-06b4-4d51-9f6c-734bdf3281ce admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.579922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.580666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.581015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.581283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.581931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.585345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.589876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.590697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.591261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.591697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.592824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.594701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.599469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1092/2151] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:26:04.604713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.608959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1060/2152] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:26:04.618635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.619361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac80d20-9df8-4155-a747-23f4c41c0db7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.624272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.625085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.625366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.625633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.626387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.628523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.636515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebd08-eb0f-4338-bd71-986d6c02fefb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.642541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1093/2153] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:04.647406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.651176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.651872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.652212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.652791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.653526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.655263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.660585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9228eb-823a-4d33-92b9-d9cbb8df9552 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.665480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.666146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.666555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.667005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] PATH_INFO: `/v3/projects/9d53e06f62ac40aa9879d548898ff7e4/users/ddfc3a6b91ba4754b80baa8ed3b0866f/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.668763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1061/2154] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:04.677646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:04.688954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2d6f-9392-40c1-9604-8077b1f42001 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:04.695091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:04.695724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.696003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.696279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] PATH_INFO: `/v3/projects/a351a370476d4b74a14b2c29e4a1a496/users/81addecc18e94b34a2d2c39a698fd381/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.706775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d53e06f62ac40aa9879d548898ff7e4, user_id=ddfc3a6b91ba4754b80baa8ed3b0866f, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.709496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d6732-27ec-4242-bf77-3803774f00d1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.717867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a351a370476d4b74a14b2c29e4a1a496, user_id=81addecc18e94b34a2d2c39a698fd381, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:04.720892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576eea4-7426-4ba1-9fc8-9aa5b1c0743c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:04.734147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1094/2155] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:04 2022] PUT /identity/v3/projects/9d53e06f62ac40aa9879d548898ff7e4/users/ddfc3a6b91ba4754b80baa8ed3b0866f/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:04.753731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.755479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.756184 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.761775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1062/2156] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:04 2022] PUT /identity/v3/projects/a351a370476d4b74a14b2c29e4a1a496/users/81addecc18e94b34a2d2c39a698fd381/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:04.770729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:04.771006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:04.771272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:04.840279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:04.843747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:04.845257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] MFA Rules not processed for user `ddfc3a6b91ba4754b80baa8ed3b0866f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:04.846653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:04.964872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:05.006641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1063/2157] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:05.028665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:05.031654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:05.072490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1095/2158] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:05.077339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:05.084661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:05.177653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4b968-e4da-4009-9063-5c9d655c246e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:05.183549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:05.184929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:05.185324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:05.185694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:05.191883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:05.229162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:05.242082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae217fac-f12f-4c7e-81aa-ae457ce0aee8 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:05.248809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33334f-5887-4ee5-b8c5-d8c7fadb8130 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:05.252894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:05.254526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:05.255202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:05.255559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:05.256559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1096/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:05.284966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:05.290258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9881be-58de-4532-b9a6-3a0914752a60 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:05.322001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1064/2160] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:06.488047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.488939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.489234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.505708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.506605 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.506910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.513506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] MFA Rules not processed for user `fd862cc69267470580ef23e0223742e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:06.514869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:06.539609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] MFA Rules not processed for user `81addecc18e94b34a2d2c39a698fd381`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:06.541039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:06.578355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1097/2161] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:06.628907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1065/2162] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:06.634333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:06.646019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4786ff-1ced-4d8f-a419-c0ed7a53168d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:06.652979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:06.653589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.653865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.654238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.654942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:06.656930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:06.688796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1098/2163] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:06.692805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:06.703431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92289f7-a9da-4b9f-8f54-1a7f2d92ee4d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:06.761973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:06.762616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.762888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.763154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.763802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:06.765755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:06.794499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1066/2164] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:06 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:06.795983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:06.806738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaee39-c681-47aa-907d-12fd132fa06e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:06.811820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:06.812434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.812801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.812974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.813762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:06.815414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:06.826662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1099/2165] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:06.831141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:06.842264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff79b-ee53-4a32-b582-0d31b8c4bfc7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:06.847688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:06.848575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.848954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.849387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] PATH_INFO: `/v3/projects/89bf5cbc50ff45108ea60978c3360581/users/6d0d956904864b348d42e9f95ce16ffd/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.866636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bf5cbc50ff45108ea60978c3360581, user_id=6d0d956904864b348d42e9f95ce16ffd, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:06.869502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:06.884600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1067/2166] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:06 2022] PUT /identity/v3/projects/89bf5cbc50ff45108ea60978c3360581/users/6d0d956904864b348d42e9f95ce16ffd/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:06.888552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:06.900625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29804c32-e47b-437b-bb8e-0028e046df0a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:06.971371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:06.992245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:06.993568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:06.993849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:06.994117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:06.994802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:06.997121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73d85a-e566-4493-adca-581f74cca497 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:07.013252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1100/2167] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:07.015151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:07.031863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495ba9c-5c3a-446f-870f-643305e3a030 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:07.043792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:07.044800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:07.045282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:07.045743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] PATH_INFO: `/v3/projects/89bf5cbc50ff45108ea60978c3360581/users/6d0d956904864b348d42e9f95ce16ffd/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:07.052023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bf5cbc50ff45108ea60978c3360581, user_id=6d0d956904864b348d42e9f95ce16ffd, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:07.055453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:07.075519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1068/2168] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:07 2022] PUT /identity/v3/projects/89bf5cbc50ff45108ea60978c3360581/users/6d0d956904864b348d42e9f95ce16ffd/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:08.188826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:08.189613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:08.189613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:08.254402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] MFA Rules not processed for user `6d0d956904864b348d42e9f95ce16ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:08.259310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:08.291932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:08.364430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:08.372980 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1101/2169] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:08.387541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:08.393508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:08.434175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a416081-5337-40d5-af24-9921a6f0e8a4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:08.441008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:08.441809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:08.442090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:08.442359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] PATH_INFO: `/v3/users/ddfc3a6b91ba4754b80baa8ed3b0866f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:08.444223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfc3a6b91ba4754b80baa8ed3b0866f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:08.446557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:08.513218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846eeb4-913c-4169-b52b-77cd7722162b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:08.522269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:08.523069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:08.523386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:08.523711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:08.530794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfc3a6b91ba4754b80baa8ed3b0866f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:08.549348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfc3a6b91ba4754b80baa8ed3b0866f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:08.551034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfc3a6b91ba4754b80baa8ed3b0866f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:08.551979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfc3a6b91ba4754b80baa8ed3b0866f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:08.563715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fbf8fa82-e52a-47c8-bda1-911a390f498f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfc3a6b91ba4754b80baa8ed3b0866f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:08.585719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1069/2170] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:08 2022] DELETE /identity/v3/users/ddfc3a6b91ba4754b80baa8ed3b0866f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:08.591510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1100503-c29a-4751-b422-3a0ad805d639 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:08.591804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1100503-c29a-4751-b422-3a0ad805d639 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:08.592356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1100503-c29a-4751-b422-3a0ad805d639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:08.610439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:08.612916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:08.613589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b1100503-c29a-4751-b422-3a0ad805d639 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:08.614884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b1100503-c29a-4751-b422-3a0ad805d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:08.627781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1102/2171] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:08.686110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1070/2172] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:08.696223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:08.736773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:08.771495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227622b-f292-43dd-9633-5949eff395a9 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:08.857021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:08.857721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:08.858219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:08.858564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:08.870493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:08.872233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:08.913028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1103/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:26:09.544634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:09.545119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:09.545240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:09.566546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:09.568071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:09.621598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:09.642439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1071/2174] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:26:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:09.655967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:09.657421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:09.740245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afb645-3c6f-41bc-87ed-3791753279c4 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:09.746694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:09.747515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:09.747844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:09.748177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:09.758747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:09.760485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77290-01bc-4813-9f62-70e2260eb9be service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:09.775484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1104/2175] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:26:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:09.830446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:09.832339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:09.867696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058aea0-dee9-4178-844d-d3cc67ef6782 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:09.877158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:09.878309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:09.878660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:09.878938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] PATH_INFO: `/v3/projects/9d53e06f62ac40aa9879d548898ff7e4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:09.880382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d53e06f62ac40aa9879d548898ff7e4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:09.884905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.046313 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d53e06f62ac40aa9879d548898ff7e4', 'request_id': 'req-1cef6077-ab81-43f6-a069-ab2bee7b93f2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:10.064622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d53e06f62ac40aa9879d548898ff7e4', 'request_id': 'req-1cef6077-ab81-43f6-a069-ab2bee7b93f2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:10.075094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d53e06f62ac40aa9879d548898ff7e4', 'request_id': 'req-1cef6077-ab81-43f6-a069-ab2bee7b93f2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:10.118424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] The token cache is being invalidate because project 9d53e06f62ac40aa9879d548898ff7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:10.118753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:10.123036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:10.133322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ed3f-f62c-430f-a3c3-01dfd883c581 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.133662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ed3f-f62c-430f-a3c3-01dfd883c581 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.133899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92ed3f-f62c-430f-a3c3-01dfd883c581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.137040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1072/2176] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:09 2022] DELETE /identity/v3/projects/9d53e06f62ac40aa9879d548898ff7e4 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:10.176516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ec92ed3f-f62c-430f-a3c3-01dfd883c581 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:10.177814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ec92ed3f-f62c-430f-a3c3-01dfd883c581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:10.181221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.183508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:10.291793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef6077-ab81-43f6-a069-ab2bee7b93f2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.296724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.297770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.298374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.298695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.301689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:10.324464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1105/2177] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:10.340209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.340427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.342027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.396786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.399085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.410946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:10.412464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:10.413331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1073/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:10.448728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1106/2179] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:10.458120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.476472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0e366-8463-49a8-88a1-c1b85ec90f41 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.481854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.482621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.482836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.483072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.483710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:26:10.484032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.486076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.502630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1074/2180] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:26:10.506453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.518369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd2047-5b67-497b-a20b-578257a5125a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.523286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.523976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.524295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.524603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.525400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.527398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.533687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:10.567029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1107/2181] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:10 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:10.571622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.585571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1c741-957e-41a3-942b-e6315a88147e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.602185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:10.670745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.671476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.671917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.672135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.673191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.675005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.704197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1075/2182] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:10 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:10.707557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.720119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c21688-4e33-4c0c-81ff-ce5c80295f75 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.726014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.726961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.727431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.727765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.728554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.730383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.748784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1108/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:10.753179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.765021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebbc72-0fc1-46b4-b754-41e633f1a6b0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.769841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.770433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.770751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.771049 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.771695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.773626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d8630-592e-4455-9a69-d045f28ab155 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.791070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1076/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:26:10.792327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:10.803105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee666f-429f-49f2-bda7-b6de57ae9684 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:10.810619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:10.811486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.811611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.811817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] PATH_INFO: `/v3/projects/430c78f8d58744a0943ac7438fe2bba3/users/2ed111ba52e74fc1893b94944b086115/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.836232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430c78f8d58744a0943ac7438fe2bba3, user_id=2ed111ba52e74fc1893b94944b086115, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:10.841081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:10.873709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1109/2185] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:10 2022] PUT /identity/v3/projects/430c78f8d58744a0943ac7438fe2bba3/users/2ed111ba52e74fc1893b94944b086115/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:10.885796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6a6dd-0aee-44f3-85a1-9dce83a62f99 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:10.886225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6a6dd-0aee-44f3-85a1-9dce83a62f99 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:10.886511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6a6dd-0aee-44f3-85a1-9dce83a62f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:10.926812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56e6a6dd-0aee-44f3-85a1-9dce83a62f99 None None] MFA Rules not processed for user `2ed111ba52e74fc1893b94944b086115`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:10.928486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56e6a6dd-0aee-44f3-85a1-9dce83a62f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:11.023019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1077/2186] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:11.035918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:11.133063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c768dba-f3bf-499c-b37a-6d38c57a6291 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:11.137104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:11.138151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:11.138501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:11.138799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:11.157172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:11.158708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:11.175311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1110/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:21.019580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:21.170696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:29.355664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443eaa-4af7-420b-a066-09205ccb39d3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:29.356844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443eaa-4af7-420b-a066-09205ccb39d3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:29.358834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443eaa-4af7-420b-a066-09205ccb39d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:29.393381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d2443eaa-4af7-420b-a066-09205ccb39d3 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:29.396264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d2443eaa-4af7-420b-a066-09205ccb39d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:29.463803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1078/2188] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:26:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:29.476224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:29.477616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:29.547035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b894-ec22-4e7e-bc8f-6fcd27271138 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:29.552608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:29.553246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:29.553527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:29.553794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:29.561976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:29.563622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:29.572987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1111/2189] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:30.114686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e81ef-4c11-465f-9a34-6f1a078a64ff None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:30.115047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e81ef-4c11-465f-9a34-6f1a078a64ff None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:30.115375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e81ef-4c11-465f-9a34-6f1a078a64ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:30.135839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-423e81ef-4c11-465f-9a34-6f1a078a64ff None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:30.137030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-423e81ef-4c11-465f-9a34-6f1a078a64ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:30.143964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1079/2190] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:30.156251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:30.166940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba1874-2f87-4241-aeeb-facad05c940f service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:30.172222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:30.172814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:30.173094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:30.173366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:30.181875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:30.183521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:30.191454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1112/2191] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:30.509042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cb87-50a9-4470-b2bd-2163c9d4b5be None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:30.509577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cb87-50a9-4470-b2bd-2163c9d4b5be None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:30.509577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786cb87-50a9-4470-b2bd-2163c9d4b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:30.551399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c786cb87-50a9-4470-b2bd-2163c9d4b5be None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:30.552801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c786cb87-50a9-4470-b2bd-2163c9d4b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:30.569014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1080/2192] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:26:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:30.578278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:30.593596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570483b-124b-47d0-b8c1-eac7553b54df service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:30.606456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:30.607775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:30.608226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:30.608653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:30.626155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:30.629321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:30.651943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1113/2193] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:31.119349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7139-35ad-4932-a5ad-9fc8103d6eec None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.119928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7139-35ad-4932-a5ad-9fc8103d6eec None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.120411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed7139-35ad-4932-a5ad-9fc8103d6eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.153435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2bed7139-35ad-4932-a5ad-9fc8103d6eec None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:31.155179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2bed7139-35ad-4932-a5ad-9fc8103d6eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:31.170532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1081/2194] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 06:26:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:31.183687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:31.194445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1394a-2782-48e3-8ea4-5ef4631f48ce None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.194823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1394a-2782-48e3-8ea4-5ef4631f48ce None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.195161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1394a-2782-48e3-8ea4-5ef4631f48ce None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.197329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c669b6-f93f-4a58-aec7-708329bd086e service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:31.199219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1082/2195] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:26:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:26:31.204791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:31.205796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.206414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.207613 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.231245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:31.235104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:31.254615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819702a-e39f-4922-84cb-727803e1415e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.255193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819702a-e39f-4922-84cb-727803e1415e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.255686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819702a-e39f-4922-84cb-727803e1415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.264504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1114/2196] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:26:31.288908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1819702a-e39f-4922-84cb-727803e1415e None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:31.290246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1819702a-e39f-4922-84cb-727803e1415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:31.496692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1083/2197] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:26:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:31.540064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:31.543436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:31.568776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d874c3-e309-468f-9e10-ac5b2eced895 service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:31.575997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:31.576493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.576855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.577044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.588174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:31.590470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c649b-f55b-4361-814c-a83326e7e222 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:31.603739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1115/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:31.674789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819702a-e39f-4922-84cb-727803e1415e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:31.784643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819702a-e39f-4922-84cb-727803e1415e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:31.791519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:31.792393 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.792853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.793551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] PATH_INFO: `/v3/users/2ed111ba52e74fc1893b94944b086115` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:31.795213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed111ba52e74fc1893b94944b086115)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:31.811389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:31.846536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:31.902044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed111ba52e74fc1893b94944b086115'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:31.914221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed111ba52e74fc1893b94944b086115'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:31.915793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed111ba52e74fc1893b94944b086115'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:31.916587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed111ba52e74fc1893b94944b086115'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:31.946389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed111ba52e74fc1893b94944b086115'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:31.986354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1084/2199] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:31 2022] DELETE /identity/v3/users/2ed111ba52e74fc1893b94944b086115 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:31.988230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:31.988720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:31.989157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:32.041790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:32.043097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:32.120835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1116/2200] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:32.139567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:32.141291 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:32.243468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418f4d0f-9a7f-4ba4-8632-5d82e0a2302d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:32.247099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:32.247699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:32.247975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:32.248242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:32.258487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:32.262181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:32.271828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1085/2201] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:32.775798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:32.777260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:32.791617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a2bb-032d-48e1-ae06-666dd12c5d5b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:32.812629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:32.813901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:32.814759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:32.815217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] PATH_INFO: `/v3/projects/430c78f8d58744a0943ac7438fe2bba3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:32.817562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430c78f8d58744a0943ac7438fe2bba3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:32.822540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:32.860084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:32.861504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:32.886116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4820bfc-b555-4a15-80f1-16368321f809 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:32.904191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:32.906616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:32.907317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:32.907596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:32.907871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] PATH_INFO: `/v3/users/81addecc18e94b34a2d2c39a698fd381` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:32.909620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=81addecc18e94b34a2d2c39a698fd381)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:32.912147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:32.923144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430c78f8d58744a0943ac7438fe2bba3', 'request_id': 'req-3920b9ab-6085-4665-a683-656bc85fe7f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:32.968856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430c78f8d58744a0943ac7438fe2bba3', 'request_id': 'req-3920b9ab-6085-4665-a683-656bc85fe7f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:32.987046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430c78f8d58744a0943ac7438fe2bba3', 'request_id': 'req-3920b9ab-6085-4665-a683-656bc85fe7f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.007333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] The token cache is being invalidate because project 430c78f8d58744a0943ac7438fe2bba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:33.007836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.008731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3920b9ab-6085-4665-a683-656bc85fe7f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.015662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1117/2202] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:32 2022] DELETE /identity/v3/projects/430c78f8d58744a0943ac7438fe2bba3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:33.016343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81addecc18e94b34a2d2c39a698fd381'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.017518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.018021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.018453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.035118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81addecc18e94b34a2d2c39a698fd381'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.039206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81addecc18e94b34a2d2c39a698fd381'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.043960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81addecc18e94b34a2d2c39a698fd381'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.059214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5c84a37d-bc49-4c0e-8916-f477c7303afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81addecc18e94b34a2d2c39a698fd381'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.063597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:33.065015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.132556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1086/2203] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:32 2022] DELETE /identity/v3/users/81addecc18e94b34a2d2c39a698fd381 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:33.196245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1118/2204] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:33.220089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.220326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.220619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.256311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:33.261626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.285354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:33.352309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1087/2205] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:33.353121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.363401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a6f5-bbf5-4bab-9c17-5a7a02277f60 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.374175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.374857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.375227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.375679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.376333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:26:33.376780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.378748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.403458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1119/2206] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:26:33.408477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.423364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19208b-d78e-4e2a-a0af-4c3d4c7c7024 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.431230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.431922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.432371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.432739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.434003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.437562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.482068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1088/2207] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:33 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:26:33.487159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.494286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.497242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.507551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b43fc3-5ece-4a78-87cd-ac0e1beeeca7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.587368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba83dc-b098-492e-8643-5b4f039abc0b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.591885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.592668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.592985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.593263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] PATH_INFO: `/v3/projects/a351a370476d4b74a14b2c29e4a1a496` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.594395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a351a370476d4b74a14b2c29e4a1a496)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.596353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.604465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.605196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.605533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.605887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.606618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.608614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.634283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1120/2208] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:33 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:33.638675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.649904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8155c-91fc-4086-b44e-eaa5d7dbf403 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.662414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a351a370476d4b74a14b2c29e4a1a496', 'request_id': 'req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.671077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a351a370476d4b74a14b2c29e4a1a496', 'request_id': 'req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.683764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a351a370476d4b74a14b2c29e4a1a496', 'request_id': 'req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.686907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:33.701444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] The token cache is being invalidate because project a351a370476d4b74a14b2c29e4a1a496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:33.701742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.702411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.706672 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1089/2209] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:33 2022] DELETE /identity/v3/projects/a351a370476d4b74a14b2c29e4a1a496 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:33.708610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.711214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.715784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.716393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.716669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.716938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.717583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.719260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.723602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c038-dfc6-47b9-8d7b-e2d0db4cec4b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.729168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.729602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1121/2210] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:33.729885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.730201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.730541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] PATH_INFO: `/v3/users/6d0d956904864b348d42e9f95ce16ffd` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.731880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d0d956904864b348d42e9f95ce16ffd)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.733427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.734428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.735231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.747754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8fb2-932e-4721-9c6f-7525b16d688c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.752351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.752936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.753213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.753483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.754112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.755770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.758287 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:33.766337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1122/2211] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:26:33.770939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.781047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a24a8-7285-4c6b-bfa7-253779802e5e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:33.785874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:33.786518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.786845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.787119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] PATH_INFO: `/v3/projects/d0adbb8b987449f08ee39d325424b1da/users/c2b511ca42e6427aa2f1627c87ad3fad/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.788432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d0d956904864b348d42e9f95ce16ffd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.796506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d0d956904864b348d42e9f95ce16ffd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.797520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d0d956904864b348d42e9f95ce16ffd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.798226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d0d956904864b348d42e9f95ce16ffd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.803752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0adbb8b987449f08ee39d325424b1da, user_id=c2b511ca42e6427aa2f1627c87ad3fad, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:33.806414 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604860a-cb37-4194-bc81-444382e03310 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:33.807040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d0d956904864b348d42e9f95ce16ffd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:33.828332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1123/2212] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:33 2022] PUT /identity/v3/projects/d0adbb8b987449f08ee39d325424b1da/users/c2b511ca42e6427aa2f1627c87ad3fad/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:33.829912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:33.830341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:33.831050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1090/2213] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:33 2022] DELETE /identity/v3/users/6d0d956904864b348d42e9f95ce16ffd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:33.831945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:33.854322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:33.856224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:33.930662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1124/2214] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:33.940879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:33.942410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:34.018974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44a7b1-eea8-46c7-bceb-862458993be8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:34.024213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:34.025034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:34.025318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:34.025595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:34.038081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:34.039957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:34.047690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:34.049333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:34.057564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1091/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:34.068209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3591332-cb07-4508-9e97-b6de503779a3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:34.075005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:34.075904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:34.076235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:34.076443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] PATH_INFO: `/v3/projects/89bf5cbc50ff45108ea60978c3360581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:34.077798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89bf5cbc50ff45108ea60978c3360581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:34.079927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:34.132777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89bf5cbc50ff45108ea60978c3360581', 'request_id': 'req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.138872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89bf5cbc50ff45108ea60978c3360581', 'request_id': 'req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.146874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89bf5cbc50ff45108ea60978c3360581', 'request_id': 'req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.166195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] The token cache is being invalidate because project 89bf5cbc50ff45108ea60978c3360581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:34.166536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.167288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8d9cf750-ccd8-46c0-ad7d-decb74870fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.169389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1125/2216] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:34 2022] DELETE /identity/v3/projects/89bf5cbc50ff45108ea60978c3360581 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:34.318597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:34.320365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:34.338026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c43207-0fd3-43ec-a5ed-15b413198096 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:34.420962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:34.421514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:34.421787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:34.422066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] PATH_INFO: `/v3/users/4d1f91620b8e4d87b7ce4a015b0d447d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:34.423218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1f91620b8e4d87b7ce4a015b0d447d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:34.425245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:34.509063 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:34.517694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1f91620b8e4d87b7ce4a015b0d447d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.528909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1f91620b8e4d87b7ce4a015b0d447d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.529655 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1f91620b8e4d87b7ce4a015b0d447d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.530633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1f91620b8e4d87b7ce4a015b0d447d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.542013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1f91620b8e4d87b7ce4a015b0d447d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:34.580822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1092/2217] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:26:34 2022] DELETE /identity/v3/users/4d1f91620b8e4d87b7ce4a015b0d447d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:34.585490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:34.585772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:34.586044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:34.608271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:34.609615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:34.639428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:34.691309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1126/2218] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:34.714535 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:34.716359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:34.813497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ba9c9-4a38-4047-a0c4-0275b8a18ac3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:34.817371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:34.818109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:34.818492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:34.818972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:34.830968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:34.832762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536830ae-6426-4067-be44-0621cece55ff service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:34.844147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1093/2219] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:35.264819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.266500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:35.295370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272aeb2d-11eb-4953-93ea-f0df4410207e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.300645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.301328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.301634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.302671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] PATH_INFO: `/v3/projects/89b44de54291465a8b30343a11d72f78` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.304750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b44de54291465a8b30343a11d72f78)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.306442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.372862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b44de54291465a8b30343a11d72f78', 'request_id': 'req-59d60108-37d5-4e09-a3dc-9c8487c95cf5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:35.379446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b44de54291465a8b30343a11d72f78', 'request_id': 'req-59d60108-37d5-4e09-a3dc-9c8487c95cf5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:35.393300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b44de54291465a8b30343a11d72f78', 'request_id': 'req-59d60108-37d5-4e09-a3dc-9c8487c95cf5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:35.410832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] The token cache is being invalidate because project 89b44de54291465a8b30343a11d72f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:26:35.411760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:35.412620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-59d60108-37d5-4e09-a3dc-9c8487c95cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:26:35.415180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1127/2220] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:26:35 2022] DELETE /identity/v3/projects/89b44de54291465a8b30343a11d72f78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:35.420022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.420551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.420551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.449710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:35.451554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:35.541750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1094/2221] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:35.549956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.550468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.551506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.581260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:35.582650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:35.596747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1128/2222] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:35.598048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.617705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88458a-9c7b-44ba-9303-df8a7034cb92 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.631152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.635246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.635686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.636983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.638410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:26:35.640996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.644307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.669013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1095/2223] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:26:35.673164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.685811 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebe00e-cdba-4967-98dd-b70170232bf0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.695135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.696087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.696523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.696948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.697772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.700307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.743482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1129/2224] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:26:35.748620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.765977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81336396-5dd1-470b-8013-fb1b84d1bc51 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.827326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:35.854873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.855638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.856356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.856356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.857095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.859148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.886414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1096/2225] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:35 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:35.891486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.904096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d18ea-636d-4413-b311-0043a3b9d7b7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.910142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.910807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.911143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.911418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.912068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.914946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.930117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1130/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:35.944110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:35.967997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a48a5-8ea6-45a8-9bfc-ecc5ce65d069 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:35.974167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:35.974825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:35.975727 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:35.976031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:35.976705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:35.978376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aed837-74c4-4d09-ae75-957bdb833ad9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:35.991432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1097/2227] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:35.995507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:36.007068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e1abd-65b4-4c63-b7ed-e7fa9d1fa82b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:36.015274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:36.015960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:36.016238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:36.016510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] PATH_INFO: `/v3/projects/1d431bdc9a2244dcb1bb589dee792d02/users/e702f6ad9e38492497e0ae33abb9aae7/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:36.046068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d431bdc9a2244dcb1bb589dee792d02, user_id=e702f6ad9e38492497e0ae33abb9aae7, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:36.048756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:36.071150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1131/2228] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:35 2022] PUT /identity/v3/projects/1d431bdc9a2244dcb1bb589dee792d02/users/e702f6ad9e38492497e0ae33abb9aae7/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:26:36.076518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41224656-b134-4dee-a01e-b1998d7eb722 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:36.076935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41224656-b134-4dee-a01e-b1998d7eb722 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:36.077208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41224656-b134-4dee-a01e-b1998d7eb722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:36.129221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41224656-b134-4dee-a01e-b1998d7eb722 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:36.131376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41224656-b134-4dee-a01e-b1998d7eb722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:36.217953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1098/2229] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:36.230268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:36.233163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:36.299628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:36.317210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abc45b-29c6-44be-a514-cc7aa49ec258 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:36.323426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:36.323718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:36.324849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:36.325128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:36.339757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:36.341114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:36.353380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1132/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:37.349157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728961-aaf2-4c47-b135-09dd6ddb2abe None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:37.349701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728961-aaf2-4c47-b135-09dd6ddb2abe None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:37.349701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728961-aaf2-4c47-b135-09dd6ddb2abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:37.439034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7f728961-aaf2-4c47-b135-09dd6ddb2abe None None] MFA Rules not processed for user `e702f6ad9e38492497e0ae33abb9aae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:37.440233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7f728961-aaf2-4c47-b135-09dd6ddb2abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:37.640917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1099/2231] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:37.655679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:37.659467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:37.792368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681093d-a1e2-470d-943e-e56de606ec38 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:37.796684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:37.797312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:37.797589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:37.797856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:37.808525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:37.810571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265ea13-2dad-46fb-acb4-fcdfaae1a2d4 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:37.835115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1133/2232] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:39.564522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:39.565093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:39.565093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:39.605455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] MFA Rules not processed for user `e702f6ad9e38492497e0ae33abb9aae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:39.606872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:39.624106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1100/2233] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:41.468936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:41.470432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:41.471327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:41.556580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] MFA Rules not processed for user `c2b511ca42e6427aa2f1627c87ad3fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:41.557842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:41.837995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1134/2234] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:41.841225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:41.844447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:41.925905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1217-5090-4cfd-bcf9-cf594510b31a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:41.948718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:41.949351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:41.949804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:41.950613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:41.951807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:41.967506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.111582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1101/2235] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:26:41 2022] POST /identity/v3/projects => generated 371 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:42.113006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:42.140146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b028c-dd84-44ca-b6ed-67d9bac49398 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:42.288007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:42.288491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.288749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.289302 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.290073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:42.292938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.330967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1135/2236] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:26:42 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:26:42.333083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:42.366243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d090b-38b0-44e8-9cb0-2263c8c8518d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:42.382173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:42.382836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.383358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.383770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.384546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:42.386284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.403653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1102/2237] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:26:42.409157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:42.425915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb2f08-c10e-4222-918f-16ad090c6357 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:42.432118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:42.432588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.432866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.433139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] PATH_INFO: `/v3/projects/a4df4f13709a46a5a5891e28798c8b19/users/dcab7516ea6640d1a4a8e3231c518ea7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.445292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:42.488791 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4df4f13709a46a5a5891e28798c8b19, user_id=dcab7516ea6640d1a4a8e3231c518ea7, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:42.497522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994f47-c575-465a-b3fc-719f82d292f2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.536158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1136/2238] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:42 2022] PUT /identity/v3/projects/a4df4f13709a46a5a5891e28798c8b19/users/dcab7516ea6640d1a4a8e3231c518ea7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:42.538154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:42.552215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2e6b1-62d9-402c-b5a2-779453d3601d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:42.586764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:42.628620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2bfa-eac8-43b7-abf7-5527c6b8c229 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.629058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2bfa-eac8-43b7-abf7-5527c6b8c229 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.629436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce2bfa-eac8-43b7-abf7-5527c6b8c229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.659450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dcce2bfa-eac8-43b7-abf7-5527c6b8c229 None None] MFA Rules not processed for user `7a6b8fbd31464e4d99449c74b4d1dda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:42.661604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dcce2bfa-eac8-43b7-abf7-5527c6b8c229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:42.702453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:42.703926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.704200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.704476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.705201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:42.707361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.719897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1103/2239] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:26:42.724030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:42.742237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc5d51-5fa9-4d4f-88ba-21b11ec3a8b3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:42.749267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:42.750419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:42.750856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:42.751258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] PATH_INFO: `/v3/projects/a4df4f13709a46a5a5891e28798c8b19/users/dcab7516ea6640d1a4a8e3231c518ea7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:42.753633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4df4f13709a46a5a5891e28798c8b19, user_id=dcab7516ea6640d1a4a8e3231c518ea7, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:42.763741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:42.781419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1137/2240] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:26:42.803295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1104/2241] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:26:42 2022] PUT /identity/v3/projects/a4df4f13709a46a5a5891e28798c8b19/users/dcab7516ea6640d1a4a8e3231c518ea7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:26:43.019926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27f6b-091b-482a-b28f-59bf675ae568 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:43.020210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27f6b-091b-482a-b28f-59bf675ae568 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:43.020479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27f6b-091b-482a-b28f-59bf675ae568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:43.042072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-55f27f6b-091b-482a-b28f-59bf675ae568 None None] MFA Rules not processed for user `d85160b01d824bc3a32b425fab98d2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:43.043322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-55f27f6b-091b-482a-b28f-59bf675ae568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:43.140497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1138/2242] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:50.400437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:50.402271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:50.493171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ba6c3-c66a-4a41-928a-f89c593baf8d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:50.497682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:50.498308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:50.498598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:50.498903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:50.500767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:50.633477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:50.635461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:50.696937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1105/2243] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:51.504926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0c15d-d00b-4b40-9076-c3edbb4b5d5e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:51.506038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0c15d-d00b-4b40-9076-c3edbb4b5d5e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:51.508786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0c15d-d00b-4b40-9076-c3edbb4b5d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:51.557413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-55a0c15d-d00b-4b40-9076-c3edbb4b5d5e None None] MFA Rules not processed for user `dcab7516ea6640d1a4a8e3231c518ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:51.559253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-55a0c15d-d00b-4b40-9076-c3edbb4b5d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:51.734018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1139/2244] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:26:51.751067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:26:51.752518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:51.824698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:26:51.869109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9037f8-65d2-46eb-bca9-63d459bd34c1 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:26:51.879595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:26:51.882107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:51.882107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:51.882107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:51.972916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:26:51.975036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:26:51.992874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1106/2245] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:26:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:26:53.723946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:26:53.725063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:26:53.725408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:26:53.753676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] MFA Rules not processed for user `c2b511ca42e6427aa2f1627c87ad3fad`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:26:53.756534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:26:53.770103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1140/2246] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:26:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:27:01.986848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=33 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:03.741936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:25.461461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:27:25.464961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:27:25.559349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951cdac-ccd2-4f6c-8865-cb4b739c78f1 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:27:25.564469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:27:25.565074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:27:25.565355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:27:25.565628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:27:25.568141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:27:25.582709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:27:25.584506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274e763-9b7d-4112-8f9d-7dd62e951679 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:27:25.605305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1107/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:27:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:27:35.592985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=32 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:27:43.671421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:27:43.680764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6c39-071d-44f9-8aed-38cc00cbeab6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:27:43.684306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:27:43.684899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:27:43.685175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:27:43.685443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:27:43.696464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:27:43.697902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e3d49-d5e5-42c1-b522-6f0015d58c91 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:27:43.709831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1141/2248] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:27:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:27:53.703177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:07.306046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:07.306542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:07.306778 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:07.317443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ecda-3b4c-4e23-b7de-26c1d645cf34 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:07.317900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ecda-3b4c-4e23-b7de-26c1d645cf34 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:07.318072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85ecda-3b4c-4e23-b7de-26c1d645cf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:07.326884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] MFA Rules not processed for user `e702f6ad9e38492497e0ae33abb9aae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:28:07.328107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:07.337609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8b85ecda-3b4c-4e23-b7de-26c1d645cf34 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:28:07.338834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b85ecda-3b4c-4e23-b7de-26c1d645cf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:07.384979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1108/2249] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:28:07.397847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1142/2250] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 06:28:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:28:07.446133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:28:07.447603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:07.524223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb26c-1986-4a51-9e4c-20298194be02 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:28:07.529884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:28:07.530490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:07.530794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:07.531064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:07.539797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:28:07.541365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5ba07-6dcc-43e0-acf7-33cee6b641f5 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:28:07.549669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1109/2251] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 06:28:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 06:28:17.390505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=25 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:17.546126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=57 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:24.876326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:24.876902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:24.877136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:24.880567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1143/2252] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:28:24.887816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bcb3e-a837-473e-84f2-94070f9db8e7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:24.888355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bcb3e-a837-473e-84f2-94070f9db8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:24.888440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bcb3e-a837-473e-84f2-94070f9db8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:24.910139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dd5bcb3e-a837-473e-84f2-94070f9db8e7 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:28:24.911524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dd5bcb3e-a837-473e-84f2-94070f9db8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:24.921036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1110/2253] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:28:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:28:24.933055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:28:24.934800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:25.001434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624ab4-3bc8-4381-86d4-e9b7f606da75 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:28:25.006958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:28:25.007687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.007984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.008274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.019855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:28:25.021593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec30128-50d3-4237-abd9-e1e9c3b5bd92 service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:28:25.036845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1144/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:28:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:28:25.140910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22343d45-63c2-42d7-a5b2-2cf4997f0943 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.141491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22343d45-63c2-42d7-a5b2-2cf4997f0943 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.141770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22343d45-63c2-42d7-a5b2-2cf4997f0943 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.143200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1111/2255] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 06:28:25.151065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc858e-6c10-4b57-9455-5a814959cad2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.151352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc858e-6c10-4b57-9455-5a814959cad2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.151890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc858e-6c10-4b57-9455-5a814959cad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.175347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8ecc858e-6c10-4b57-9455-5a814959cad2 None None] MFA Rules not processed for user `8e742daf773049bf814cde4544b8a673`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:28:25.176591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8ecc858e-6c10-4b57-9455-5a814959cad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:25.184429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1145/2256] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:28:25.199167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22343d45-63c2-42d7-a5b2-2cf4997f0943 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:28:25.217553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22343d45-63c2-42d7-a5b2-2cf4997f0943 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:28:25.224284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:28:25.224892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.225168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.225437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.242424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:28:25.243995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eca16-2e05-4f03-bc7c-3a7f3494f5d8 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:28:25.255670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1112/2257] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:28:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:28:25.347823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404435-6b85-45e5-8bbc-dbb75c290acb None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.348209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404435-6b85-45e5-8bbc-dbb75c290acb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.348539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27404435-6b85-45e5-8bbc-dbb75c290acb None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.349967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1146/2258] 10.222.0.20 () {54 vars in 921 bytes} [Sun Sep 18 06:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 18 06:28:25.356676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.356961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.357235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.384097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] MFA Rules not processed for user `04ddfa59d1834ba3a7f1f27e963b090e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:28:25.385577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:28:25.397904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1113/2259] 10.222.0.20 () {58 vars in 1018 bytes} [Sun Sep 18 06:28:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:28:25.412289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27404435-6b85-45e5-8bbc-dbb75c290acb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:28:25.424431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27404435-6b85-45e5-8bbc-dbb75c290acb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:28:25.429203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:28:25.430150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:28:25.430703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:28:25.431198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:28:25.443760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:28:25.445323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:28:25.457322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1147/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:28:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:28:35.377970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=12 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:28:35.452089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=35 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:14.170565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:14.180356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a719a-18f3-491d-9af7-35d3fd05c858 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:14.183850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:14.184432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:14.184706 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:14.184973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:14.186673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:14.200080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:14.201493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:14.212159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1114/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:15.186078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:15.194694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd81924-f898-4764-b1ae-54f2d49f00ba service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:15.198252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:15.198873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:15.199150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:15.199424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:15.201163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:15.215399 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:15.217013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f94e0-4810-4518-b626-67173c8e8e78 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:15.229417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1148/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:19.911466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:19.920201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2d034-44cc-4cba-93b3-a2170764bbd6 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:19.923935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:19.924521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:19.925455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:19.925725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:19.927625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:19.986004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:19.988129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507510e-361a-46eb-8983-9a4a533a3937 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:19.997167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1115/2263] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:29:19 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:25.187407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:25.188614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:25.189045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:25.212408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] MFA Rules not processed for user `7a6b8fbd31464e4d99449c74b4d1dda5`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:29:25.214079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:25.228469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1149/2264] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:29:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:29:27.668819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:27.669379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:27.669462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:27.689252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] MFA Rules not processed for user `6c945a3e6a4946559aeadc63566d29f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:29:27.691195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:27.747555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1116/2265] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:29:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:29:28.097211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:28.106997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07466a-cf26-40c7-a920-f6521f6b70c6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:28.111985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:28.113115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:28.113382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:28.113649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:28.115733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:28.127256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:28.129018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:28.137138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1150/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:28.457065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:28.458493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:28.474180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b33046-1db4-4a67-bdc2-bf0aa0e069d3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:28.480183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:28.480795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:28.481090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:28.481351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:28.483101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:28.495452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:28.497284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:28.508127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1117/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:29:30.973046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:30.983614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8aeb2-a2c0-4102-a82a-d187750e58b0 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:30.988508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:30.989308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:30.989596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:30.989865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:30.991774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:31.003993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:31.006329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:31.020155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1151/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:38.503708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=65 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:40.622731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:40.635280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9021c-1a3b-48bf-a6c0-83e42b8cd7cb service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:40.641249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:40.641842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:40.642119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:40.642965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:40.644702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:40.656793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:40.658997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:40.669499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1118/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:41.012163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=21 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:42.357549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:29:42.359063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:42.375955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf2200-854a-46a2-a157-27fe9fbc96f5 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:29:42.381871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:29:42.382462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:29:42.382813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:29:42.383082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:29:42.384772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:29:42.395234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:29:42.396803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:29:42.405564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1152/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:29:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:29:50.665081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:29:52.401469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:11.286213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:30:11.299363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1c305-1ed1-40ef-848a-68f05e629d89 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:30:11.305336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:30:11.305944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:30:11.306250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:30:11.306662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:30:11.308485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:30:11.321028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:30:11.322621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:30:11.333999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1119/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:30:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:30:16.758647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:30:16.769589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe0995-81a1-47f0-af02-a9854a83f6f8 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:30:16.774953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:30:16.775543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:30:16.775817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:30:16.776086 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:30:16.777763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:30:16.788202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:30:16.789767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:30:16.799133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1153/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:30:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:30:21.328587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:26.794431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:44.339239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:30:44.351769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f32ede-efd5-4b8d-8538-1a8afd15b8eb service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:30:44.357224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:30:44.357805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:30:44.358079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:30:44.358350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:30:44.360075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:30:44.419644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:30:44.421144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:30:44.430704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1120/2273] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:30:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:30:54.426493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=28 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:30:58.942073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:30:58.953231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120edfbf-6050-4c12-a2f6-361d03db0f4c service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:30:58.959160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:30:58.959749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:30:58.960032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:30:58.960301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:30:58.962006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:30:59.019827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:30:59.021405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:30:59.030170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1154/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:30:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:31:09.026771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=28 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:31:51.572911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:31:51.584321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38ddee-d6c4-4fba-831c-4dc226bf6a5d service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:31:51.589966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:31:51.590564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:31:51.590872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:31:51.591147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:31:51.599480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:31:51.601060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:31:51.610565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1121/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:31:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:32:01.605912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:07.442364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:32:07.443846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:32:07.499001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c27048-2fff-4697-8e27-cab9776b2e73 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:32:07.503388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:32:07.503822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:32:07.504097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:32:07.504365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:32:07.514944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:32:07.516953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:32:07.525708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1155/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:32:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:32:17.521697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=26 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:32:29.035551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:32:29.108612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebeb41-f70f-477c-b2cd-2c1cce4d375d service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:32:29.114513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:32:29.115145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:32:29.115418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:32:29.115684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:32:29.124850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:32:29.126437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:32:29.136915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1122/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:32:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:32:39.131935 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=27 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:15.073502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:34:15.102375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e31af9-ef33-4c96-9630-56c75b9f8423 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:34:15.105948 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:34:15.106544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:34:15.106848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:34:15.107114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:34:15.123556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:34:15.124967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:34:15.145713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1156/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:34:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:34:16.320906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:34:16.330340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378cd1a-05d4-4647-9889-4577247feed8 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:34:16.333983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:34:16.334568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:34:16.334872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:34:16.335143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:34:16.345798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:34:16.347249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:34:16.358635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1123/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:34:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:34:20.837420 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:34:20.848287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f45f7-210a-4fdd-8285-0f5f988cf1aa service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:34:20.853652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:34:20.854231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:34:20.854505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:34:20.854798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:34:20.862689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:34:20.864264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:34:20.891451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1157/2280] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:34:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:34:26.353214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:30.888931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=35 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:34:36.740572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:34:36.749344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60062f3-407c-4d16-bf2f-2366102eade1 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:34:36.752900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:34:36.753474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:34:36.753748 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:34:36.754011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:34:36.760904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:34:36.762847 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:34:36.770401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1124/2281] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:34:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:34:46.767091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:22.047079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:35:22.065815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065e35a-45a2-460a-b67f-0551c4237ab7 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:35:22.069400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:35:22.070000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:35:22.070281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:35:22.070552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:35:22.078851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:35:22.080281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:35:22.107746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1158/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:35:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:35:22.168881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:35:22.178055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf763f-2242-4dac-af3f-545f1ad95aaf service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:35:22.181719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:35:22.182301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:35:22.182591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:35:22.182870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:35:22.191210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:35:22.192646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:35:22.221683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1125/2283] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:35:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:35:32.105073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=17 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:35:32.218845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=13 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:07.661803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:36:07.670485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1190b6-5caf-45a8-8cd3-ca8503c0de24 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:36:07.674071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:36:07.674691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:36:07.674972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:36:07.675238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:36:07.684323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:36:07.685776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:36:07.712434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1159/2284] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:36:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:36:07.756676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:36:07.763791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faff22-6f20-4a14-bc2d-a7fc36d838e9 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:36:07.767843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:36:07.768656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:36:07.769137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:36:07.769608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:36:07.778193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:36:07.779766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:36:07.806270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1126/2285] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:36:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:36:17.709881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=13 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:36:17.803635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=13 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:37:22.451744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:37:22.453959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:37:22.488354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce28f7-212b-4847-a2b2-a8cc5cf2cca5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:37:22.492014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:37:22.492616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:37:22.492896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:37:22.493163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:37:22.495075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:37:22.528130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:37:22.529577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:37:22.547521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1160/2286] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:37:22 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:37:32.544873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=28 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:37:34.014380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:37:34.079690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e699b8d-4d4a-4d7f-8b2e-262490ca04ce service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:37:34.084716 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:37:34.085306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:37:34.085585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:37:34.085859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:37:34.087742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:37:34.111569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:37:34.113144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:37:34.122876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1127/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:37:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:37:44.118838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=37 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:38:08.063911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:38:08.080166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:38:08.081848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:38:08.145453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986accff-7fa4-41de-bf34-f490a4f55343 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:38:08.152830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:38:08.153437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:38:08.153714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:38:08.153985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:38:08.156277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:38:08.188689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008afc9-a094-4400-a263-5ac8d9dde585 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:38:08.194682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:38:08.195280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:38:08.195557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:38:08.195829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:38:08.205345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:38:08.207039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:38:08.217543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1128/2288] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:38:08.218146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:38:08.227478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b62ad05-fc9c-41bf-bcc2-e079b0b6af2d service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:38:08.234070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:38:08.234892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:38:08.235288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:38:08.235601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:38:08.237878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:38:08.239739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:38:08.244507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:38:08.245977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:38:08.261677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1161/2289] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:38:08.272367 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1129/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:38:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:38:18.257451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=69 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:38:18.268172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=49 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:22.855524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:39:22.865215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18516379-1be4-4101-9ad5-80b8217d5fd4 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:39:22.868888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:39:22.869409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:39:22.869681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:39:22.869945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:39:22.871877 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:39:22.887107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:39:22.888563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:39:22.899383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1162/2291] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:39:22.948962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:39:22.961441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbc2a0-4922-4dca-9603-d8a920831aad service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:39:22.966540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:39:22.967176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:39:22.967451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:39:22.967720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:39:22.969740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:39:22.983333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:39:22.984830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:39:22.996455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1130/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:39:23.087410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:39:23.096371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d5fa7-4934-4aa7-bc2f-5d55ab687787 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:39:23.099923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:39:23.100516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:39:23.100794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:39:23.101066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:39:23.102943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:39:23.174282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:39:23.176344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:39:23.185292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1163/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:39:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:39:32.991213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:33.181020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=39 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:39:40.730924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:39:40.732540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:39:40.805422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347a357-b65e-40e5-87d3-2b5e8fef9629 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:39:40.808980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:39:40.809566 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:39:40.809835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:39:40.810098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:39:40.811907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:39:40.821027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:39:40.822465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:39:40.830252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1131/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:39:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:39:50.826781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=31 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:23.409405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:41:23.417886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db238d-f588-41b0-9a56-1a26047d9ece service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:41:23.423511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:41:23.424141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:41:23.424416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:41:23.424745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:41:23.426766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:41:23.497391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:41:23.498870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:41:23.508462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1164/2295] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:41:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:41:23.595918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:41:23.600896 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:41:23.603596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e1c82-96ff-4cfc-9039-0c69640af865 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:41:23.607961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:41:23.607961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:41:23.608283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:41:23.608437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:41:23.610247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:41:23.612297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd0382-2c6e-4125-b5ac-4af3a276f6fa service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:41:23.617871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:41:23.618465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:41:23.618770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:41:23.619054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:41:23.620954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:41:23.684684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:41:23.686117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:41:23.691553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:41:23.693135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:41:23.695273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1132/2296] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:41:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:41:23.704661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1165/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:41:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:41:23.938069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:41:23.939723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:41:23.955573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb20720-9b2b-4b3d-b4c1-3f138d50649c service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:41:23.961122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:41:23.961764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:41:23.962040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:41:23.962308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:41:23.964226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:41:24.034787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:41:24.036400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:41:24.045925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1133/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:41:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:41:33.699879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=66 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:41:34.041122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=66 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:42:39.005479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:42:39.007067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:42:39.059521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196e558-ed19-4311-a208-ddcbc3c1cf57 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:42:39.064945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:42:39.065532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:42:39.065807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:42:39.066075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:42:39.074178 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:42:39.075793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:42:39.086376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1166/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:42:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:42:49.082153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=23 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:43:23.959497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:43:24.012530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a229e-760a-4e44-9cdb-bf20e1c4f89a service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:43:24.016082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:43:24.016673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:43:24.016949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:43:24.017215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:43:24.071317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:43:24.072738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:43:24.081899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1134/2300] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:43:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:43:24.166962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:43:24.180008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef756ad-616d-4446-a249-8bae1b5d679d service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:43:24.186115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:43:24.186757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:43:24.187034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:43:24.187305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:43:24.196183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:43:24.197756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:43:24.210252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1167/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:43:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:43:34.077479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=48 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:43:34.206394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:44:44.745504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:44:44.773831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0de423-7995-445a-b13a-81043aa37389 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:44:44.777410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:44:44.777996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:44:44.778269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:44:44.778534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:44:44.786032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:44:44.787462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:44:44.795177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1135/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:44:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:44:54.791792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=15 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:09.509503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:46:09.511108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:46:09.541265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8667ae-ddd1-44f9-b45b-b805b25202cc service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:46:09.544917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:46:09.545550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:46:09.545826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:46:09.546092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:46:09.563713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:46:09.565816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:46:09.585405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1168/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:46:09.629854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:46:09.640620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442aa1f1-7d83-491f-97d6-ef4a40244f11 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:46:09.644467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:46:09.645189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:46:09.645462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:46:09.645861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:46:09.657082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:46:09.658495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:46:09.669551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1136/2304] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:46:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:46:19.582917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=22 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:46:19.664694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:47:24.312070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:47:24.313866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:47:24.346136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7867d4-3441-4eaf-9379-1c82b882ea47 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:47:24.350472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:47:24.351095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:47:24.351401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:47:24.351671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:47:24.359855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:47:24.361285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:47:24.388260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1169/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:47:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:47:24.433628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:47:24.441859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e30a2-2253-419d-acc2-2bd8baec3ebb service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:47:24.445411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:47:24.445984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:47:24.446262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:47:24.446528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:47:24.455298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:47:24.457297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:47:24.485447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1137/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:47:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:47:34.385730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=22 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:47:34.482761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=13 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:47:46.995870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:47:47.059986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3cb4-7784-4c25-9bb3-70bb863b1705 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:47:47.064859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:47:47.065954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:47:47.066225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:47:47.066492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:47:47.068361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:47:47.091537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:47:47.093107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:47:47.102545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1170/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:47:57.098548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=37 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:48:09.911685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:48:09.921188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57832-57a4-4741-a182-914467cd2c65 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:48:09.926433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:48:09.927191 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:48:09.927476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:48:09.927746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:48:09.936949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:48:09.938378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:48:09.966148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1138/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:48:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:48:19.963571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=13 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:49:49.706131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:49:49.707802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:49:49.772968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296bed8-e9a3-4246-894c-50732e8a15eb service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:49:49.776501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:49:49.777094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:49:49.777371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:49:49.777639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:49:49.779462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:49:49.788762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:49:49.790183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:49:49.798394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1171/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:49:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:49:59.795039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=31 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:10.080675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:50:10.089241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05ede-2529-4a62-900a-2cfa2d296f4e service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:50:10.092882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:50:10.093468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:50:10.093747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:50:10.094017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:50:10.095879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:50:10.128804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:50:10.130212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:50:10.148196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1139/2310] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:50:10 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:50:10.234660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:50:10.245839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:50:10.247416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:50:10.315125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab2f9e-6cf9-4146-a667-3a618027bd6e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:50:10.320978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:50:10.321559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:50:10.321841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:50:10.322117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:50:10.324070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:50:10.338324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16eef3-71f9-4f0c-848f-56dd7bee7bc3 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:50:10.344756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:50:10.345364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:50:10.345640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:50:10.345908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:50:10.355500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:50:10.357083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:50:10.386867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1140/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:50:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:50:10.425088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:50:10.425409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:50:10.445510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1172/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:50:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:50:10.625597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:50:10.636776 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97d932-cb3f-4a31-903a-af6a37de95c2 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:50:10.642205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:50:10.642853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:50:10.643152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:50:10.643422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:50:10.645231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:50:10.721427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:10.723709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:50:10.725574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:50:10.744641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1141/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:50:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:50:10.875001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:50:10.886551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14e8cc-6fbc-4d99-bf39-b61140e90c66 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:50:10.892038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:50:10.892627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:50:10.892901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:50:10.893168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:50:10.895169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:50:10.907902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:50:10.909495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:50:10.920206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1173/2314] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:50:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:50:20.742146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:50:20.915512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=75 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:51:09.829824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:09.831439 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:09.842965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5794408-1a3d-4037-bdb6-b905636be304 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:09.846490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:09.847111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:09.847388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:09.847680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:09.849342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:09.945854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:09.948167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:09.967561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1142/2315] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:51:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:51:12.999473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.011229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c384a5-ef4d-4d84-ab36-dbc17e7368d6 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.016834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.017541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.017919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.018221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.020075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:13.033449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.035315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.048648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1174/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:51:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:51:13.054718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.057272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:13.079625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c650b-d51e-43cd-816c-97c618c37721 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.084416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.085040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.085326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.085589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] PATH_INFO: `/v3/users/6c945a3e6a4946559aeadc63566d29f4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.086740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c945a3e6a4946559aeadc63566d29f4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.088730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.135515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c945a3e6a4946559aeadc63566d29f4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.142516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c945a3e6a4946559aeadc63566d29f4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.143151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c945a3e6a4946559aeadc63566d29f4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.143740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c945a3e6a4946559aeadc63566d29f4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.150710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d12c7a5e-d455-44fc-8288-f2dda0bc4357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c945a3e6a4946559aeadc63566d29f4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.168528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1143/2317] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:51:13 2022] DELETE /identity/v3/users/6c945a3e6a4946559aeadc63566d29f4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:51:13.350141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.361904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08e614-8187-4e78-807a-8f51e59b8f97 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.419265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.419893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.420171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.420437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] PATH_INFO: `/v3/projects/9cbc859294e9456eb2ca751279a17f52` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.421546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cbc859294e9456eb2ca751279a17f52)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.423481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.467981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cbc859294e9456eb2ca751279a17f52', 'request_id': 'req-11312a3c-d26d-43f4-b343-8b4d33c0408c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.473953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cbc859294e9456eb2ca751279a17f52', 'request_id': 'req-11312a3c-d26d-43f4-b343-8b4d33c0408c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.481792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cbc859294e9456eb2ca751279a17f52', 'request_id': 'req-11312a3c-d26d-43f4-b343-8b4d33c0408c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.492133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] The token cache is being invalidate because project 9cbc859294e9456eb2ca751279a17f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:51:13.492423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.493041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-11312a3c-d26d-43f4-b343-8b4d33c0408c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:51:13.494521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1175/2318] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:51:13 2022] DELETE /identity/v3/projects/9cbc859294e9456eb2ca751279a17f52 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:51:13.501003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.501286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.501561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.556241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:51:13.559514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:51:13.561304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:13.618943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1144/2319] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:51:13.626806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.627097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.627394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.647038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:51:13.648261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:13.656601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1176/2320] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:51:13.660703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.670820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d936c5-234e-4fd9-8c6e-1100741b0e91 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.675304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.675883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.676161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.676423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.677056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:51:13.677353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.678979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.690453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1145/2321] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:51:13.694358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.703995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f6824-6e53-4347-88ec-87fcace4b6b3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.708631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.709213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.709491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.709759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.710393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.712998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.748263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1177/2322] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:51:13 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:51:13.752297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.763087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3d746-5d40-45fb-8816-434e0efe751e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.817012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.817601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.818193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.818660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.819271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.821656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.843187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1146/2323] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:51:13 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:51:13.847304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.857687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a6afe-0274-4c39-b91e-93cb4c764f9a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.862225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.862846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.863123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.863389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.864019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.865612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.875253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1178/2324] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:51:13.878923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.889646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d516c9d-202e-425c-a031-c42fe54a2661 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.894336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.894948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.895230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.895507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.896146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.897765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69420f4f-2761-4782-ad72-eac14f4aa34f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.908738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1147/2325] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:51:13.912626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:51:13.922617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da77a9-a9bc-45ea-8fb4-c6fcc084f165 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:51:13.927240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:51:13.927885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.928164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.928439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] PATH_INFO: `/v3/projects/d1e46b3e28a34fd7881d51a4c0bec2cf/users/bb421d507bc24c3d982e0e6d02fc55e0/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.943846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e46b3e28a34fd7881d51a4c0bec2cf, user_id=bb421d507bc24c3d982e0e6d02fc55e0, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:51:13.947117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:51:13.959180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1179/2326] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:51:13 2022] PUT /identity/v3/projects/d1e46b3e28a34fd7881d51a4c0bec2cf/users/bb421d507bc24c3d982e0e6d02fc55e0/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:51:13.971481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:51:13.971766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:51:13.972044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:51:13.999704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] MFA Rules not processed for user `bb421d507bc24c3d982e0e6d02fc55e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:51:14.000940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:51:14.063866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1148/2327] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:51:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:51:23.955546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:51:24.060957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:10.434016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:10.435831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:10.495737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c694e5-4aac-4f18-b278-eba269ebad62 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:10.499300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:10.499890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:10.500164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:10.500430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:10.502133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:10.570008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:10.571485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:10.582772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1180/2328] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:52:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:52:10.630085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:10.640557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bff478-829d-4c91-a37c-baf1a3f4bcfc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:10.644152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:10.644753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:10.645032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:10.645306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:10.647071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:10.661018 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:10.662443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24369b8-b592-4496-b7ac-1823cfd728d1 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:10.673276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1149/2329] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:52:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:52:14.138548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:14.198659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f2074-31b3-4f2b-a55d-8c007c2583f3 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:14.203188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:14.203808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.204081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.204347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] PATH_INFO: `/v3/users/bb421d507bc24c3d982e0e6d02fc55e0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.205446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb421d507bc24c3d982e0e6d02fc55e0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:14.207380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:14.250032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:14.253125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb421d507bc24c3d982e0e6d02fc55e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.261320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb421d507bc24c3d982e0e6d02fc55e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.261924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb421d507bc24c3d982e0e6d02fc55e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.262545 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb421d507bc24c3d982e0e6d02fc55e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.274127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb421d507bc24c3d982e0e6d02fc55e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.286245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1181/2330] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:52:14 2022] DELETE /identity/v3/users/bb421d507bc24c3d982e0e6d02fc55e0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:52:14.290767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.291344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.291344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.310464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:52:14.311677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:14.368983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1150/2331] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:52:14.378694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:14.380074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:14.437061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec8d6f-5f7e-44ab-892d-4904441dca26 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:14.440743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:14.441345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.441621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.441891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.451456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:14.452904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8ed8b-c8fa-4cae-b9a1-58f83303b307 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:14.462620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1182/2332] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:52:14.727256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:14.728686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:14.740607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a37e82-50c6-4178-a9f8-8dd1796935e7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:14.745410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:14.746070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.746346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.746645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] PATH_INFO: `/v3/projects/d1e46b3e28a34fd7881d51a4c0bec2cf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.748281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e46b3e28a34fd7881d51a4c0bec2cf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:14.750249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:14.799669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e46b3e28a34fd7881d51a4c0bec2cf', 'request_id': 'req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.805721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e46b3e28a34fd7881d51a4c0bec2cf', 'request_id': 'req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.812790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e46b3e28a34fd7881d51a4c0bec2cf', 'request_id': 'req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.822713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] The token cache is being invalidate because project d1e46b3e28a34fd7881d51a4c0bec2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:52:14.822999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.823620 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb6952d2-5f8b-45e8-9407-15d0ee5fdd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:52:14.825763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1151/2333] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:52:14 2022] DELETE /identity/v3/projects/d1e46b3e28a34fd7881d51a4c0bec2cf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:52:14.830611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.830906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.831183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.850150 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:52:14.851383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:14.909085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1183/2334] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:52:14.916731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.917012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.917284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.937275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:52:14.938494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:14.946462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1152/2335] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:52:14.951219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:14.960923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6d4ab-9f14-4e8b-ae04-61c08995119c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:14.965502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:14.966090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.966363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.966663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:14.967308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:52:14.967606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:14.969187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:14.980448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1184/2336] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:52:14.984388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:14.993558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe31df5-406f-45b9-aa32-c2858ee4979f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:14.998657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711067-e909-4419-811f-6536e837582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:14.999233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711067-e909-4419-811f-6536e837582b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:14.999509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711067-e909-4419-811f-6536e837582b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:14.999774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b711067-e909-4419-811f-6536e837582b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.000398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711067-e909-4419-811f-6536e837582b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.002312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b711067-e909-4419-811f-6536e837582b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.029621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1153/2337] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:52:14 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:52:15.033638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.044255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b11477-24e8-479a-a6eb-a521cbcce54a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.089011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:15.099687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.100283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.100563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.100835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.101461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.103411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.125485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1185/2338] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:52:15 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:52:15.129538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711067-e909-4419-811f-6536e837582b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.140278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b711067-e909-4419-811f-6536e837582b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.144916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.145532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.145809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.146079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.146764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.148352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.158238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1154/2339] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:52:15.161967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.172542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4be42-e026-4e01-b410-6fece0c214a6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.177259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.177845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.178127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.178392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.179067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.180653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8e96c-a568-4561-9b14-b5741205d7dd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.191612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1186/2340] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:52:15.195546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.205594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf59273-8ef5-4dce-847e-2929d4053dd4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.210158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.210831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.211110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.211376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] PATH_INFO: `/v3/projects/cf5ba123a76540568cceaaa4fa54f269/users/15562cde35da48698dd6e5fb03bd69f8/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.225450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5ba123a76540568cceaaa4fa54f269, user_id=15562cde35da48698dd6e5fb03bd69f8, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.228116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.241619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1155/2341] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:52:15 2022] PUT /identity/v3/projects/cf5ba123a76540568cceaaa4fa54f269/users/15562cde35da48698dd6e5fb03bd69f8/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:52:15.252742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.253033 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.253307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.280954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] MFA Rules not processed for user `15562cde35da48698dd6e5fb03bd69f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:52:15.282170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:15.345037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1187/2342] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:52:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:52:15.349836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.359343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:15.415735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ab54c-1fe7-4c4b-af81-8e5d55f035d9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.421030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.421624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.421902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.422174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.422843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.424744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.451057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1156/2343] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:52:15 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:52:15.455214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.466808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7abc92-3517-44d5-b1dc-4a82f85d31e1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.523266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.523795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.524074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.524335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.524962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.526903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.548495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1188/2344] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:52:15 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:52:15.552670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.563485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f7e4-283a-4a59-ba92-18f1345e8f67 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.568132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.568720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.568997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.569266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.569931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.571533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.581879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1157/2345] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:52:15.585974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.596481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b1a73-8578-4bb9-802f-efbde2306c72 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.601098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.601734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.602040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.602310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] PATH_INFO: `/v3/projects/2786f19fe14c4eda8c455d0672ec11d0/users/e4892d84189c43f88b3fc26effb591f8/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.617464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2786f19fe14c4eda8c455d0672ec11d0, user_id=e4892d84189c43f88b3fc26effb591f8, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.620183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.633071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1189/2346] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:52:15 2022] PUT /identity/v3/projects/2786f19fe14c4eda8c455d0672ec11d0/users/e4892d84189c43f88b3fc26effb591f8/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:52:15.636997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.648316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58cd89-0fcc-42f6-ba4e-bc5552252824 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.703137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.703726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.704002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.704270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.704897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.706603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d9f5d-ee88-43e0-8470-40b92aabfac9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.716419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1158/2347] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:52:15.720548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:15.730569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6838eb-a642-405f-960f-141e2abfc1ce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:15.735753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:15.736388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.736668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.736939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] PATH_INFO: `/v3/projects/2786f19fe14c4eda8c455d0672ec11d0/users/e4892d84189c43f88b3fc26effb591f8/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.738884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2786f19fe14c4eda8c455d0672ec11d0, user_id=e4892d84189c43f88b3fc26effb591f8, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:15.741490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:15.751454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1190/2348] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:52:15 2022] PUT /identity/v3/projects/2786f19fe14c4eda8c455d0672ec11d0/users/e4892d84189c43f88b3fc26effb591f8/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:52:15.762828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:15.763138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:15.763412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:15.791097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] MFA Rules not processed for user `e4892d84189c43f88b3fc26effb591f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:52:15.792235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:15.873809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1159/2349] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:52:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:52:25.747938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=87,keystone:INSERT=8 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:25.870765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:52:53.967624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:52:53.969364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:54.076795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384494d-1365-4461-93a8-713cd78813d5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:52:54.081851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:52:54.082451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:52:54.082758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:52:54.083030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:52:54.084734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:52:54.096357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:52:54.097926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75776ee9-cf2f-4fac-bb38-9c8703e15d04 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:52:54.107603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1191/2350] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:52:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:53:04.103579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=49 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:15.940753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:16.023806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958daba-fff3-477f-91a7-0c00bc77c2d9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:16.028260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:16.028882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:16.029159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:16.029425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] PATH_INFO: `/v3/users/15562cde35da48698dd6e5fb03bd69f8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:16.030550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15562cde35da48698dd6e5fb03bd69f8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:16.032483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:16.079344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15562cde35da48698dd6e5fb03bd69f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.086249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15562cde35da48698dd6e5fb03bd69f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.086902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15562cde35da48698dd6e5fb03bd69f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.087520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15562cde35da48698dd6e5fb03bd69f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.094499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15562cde35da48698dd6e5fb03bd69f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.111567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1160/2351] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:53:15 2022] DELETE /identity/v3/users/15562cde35da48698dd6e5fb03bd69f8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:53:16.116100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:16.116621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:16.116621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:16.136523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:53:16.137735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:16.194462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1192/2352] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:53:16.204586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:16.205991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:16.264305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add59ee-205b-4b10-856a-f603ce95eac3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:16.267957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:16.268550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:16.268830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:16.269098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:16.277916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:16.279379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:16.288760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1161/2353] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:53:16.547554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:16.549024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:16.561162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c527b49-bd94-4ac2-86aa-a8356aaa793e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:16.566301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:16.566967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:16.567246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:16.567516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] PATH_INFO: `/v3/projects/cf5ba123a76540568cceaaa4fa54f269` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:16.568636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5ba123a76540568cceaaa4fa54f269)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:16.570542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:16.617452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5ba123a76540568cceaaa4fa54f269', 'request_id': 'req-782ce904-b691-4223-93ff-8b2b1cf5bdf4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.623507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5ba123a76540568cceaaa4fa54f269', 'request_id': 'req-782ce904-b691-4223-93ff-8b2b1cf5bdf4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.630278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5ba123a76540568cceaaa4fa54f269', 'request_id': 'req-782ce904-b691-4223-93ff-8b2b1cf5bdf4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.640562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] The token cache is being invalidate because project cf5ba123a76540568cceaaa4fa54f269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:53:16.640849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.641472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.642973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1193/2354] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:53:16 2022] DELETE /identity/v3/projects/cf5ba123a76540568cceaaa4fa54f269 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:53:16.647001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:16.651894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:16.659000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a75037-7065-4a54-9463-3fad016ab680 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:16.715382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:16.715950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:16.716226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:16.716498 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] PATH_INFO: `/v3/users/e4892d84189c43f88b3fc26effb591f8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:16.718093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4892d84189c43f88b3fc26effb591f8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:16.720071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:16.764131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4892d84189c43f88b3fc26effb591f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.771114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4892d84189c43f88b3fc26effb591f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.771722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4892d84189c43f88b3fc26effb591f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.772316 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4892d84189c43f88b3fc26effb591f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.779189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-140aef08-cbd3-4c16-adad-6cb0c216376d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4892d84189c43f88b3fc26effb591f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:16.797014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1162/2355] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:53:16 2022] DELETE /identity/v3/users/e4892d84189c43f88b3fc26effb591f8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:53:17.078677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.080141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:17.094333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ce904-b691-4223-93ff-8b2b1cf5bdf4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.150455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.151167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.151390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.151661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] PATH_INFO: `/v3/projects/2786f19fe14c4eda8c455d0672ec11d0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.152873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2786f19fe14c4eda8c455d0672ec11d0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.155467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.184330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:17.201401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2786f19fe14c4eda8c455d0672ec11d0', 'request_id': 'req-26f9e878-4c71-4ab9-b8a1-5720ef32c342'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:17.207885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2786f19fe14c4eda8c455d0672ec11d0', 'request_id': 'req-26f9e878-4c71-4ab9-b8a1-5720ef32c342'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:17.215132 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2786f19fe14c4eda8c455d0672ec11d0', 'request_id': 'req-26f9e878-4c71-4ab9-b8a1-5720ef32c342'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:17.226117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] The token cache is being invalidate because project 2786f19fe14c4eda8c455d0672ec11d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:53:17.226392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:17.227037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26f9e878-4c71-4ab9-b8a1-5720ef32c342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:53:17.228494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1194/2356] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:53:17 2022] DELETE /identity/v3/projects/2786f19fe14c4eda8c455d0672ec11d0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:53:17.236635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137792-0483-402b-8959-f05722364b3f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.237126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137792-0483-402b-8959-f05722364b3f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.237225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33137792-0483-402b-8959-f05722364b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.258184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-33137792-0483-402b-8959-f05722364b3f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:53:17.259349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-33137792-0483-402b-8959-f05722364b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:17.316182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1163/2357] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:53:17.323787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.324053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.324473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.344726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:53:17.346108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:17.354013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1195/2358] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:53:17.358107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137792-0483-402b-8959-f05722364b3f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.367788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33137792-0483-402b-8959-f05722364b3f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.372275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.372853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.373130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.373396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.374550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:53:17.374874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.376444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.388189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1164/2359] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:53:17.393010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.402951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a606a-93de-47da-925f-ce3e907dfcd9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.410544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.411535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.411984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.412495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.413149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.415098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.441759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1196/2360] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:53:17.446096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.456773 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419e0e1-07bf-4dab-b737-64988e25ec1a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.461626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:17.515689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.516208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.516487 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.516754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.517374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.519325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.539693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1165/2361] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:53:17.544671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.555026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b410d56-cc3a-4767-9553-a9a20623bc1a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.559698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.560292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.560573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.560847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.561978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.563611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.573364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1197/2362] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:53:17.577765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.587608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385cfc-8ae9-4fcf-b8e6-e4c6bb5b242a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.592330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.592942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.593218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.593488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.594162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.595945 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5620c3-b10f-4fc6-ae6b-09a2be6dc860 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.606359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1166/2363] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:53:17.610953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.620898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7af28d-2c92-4f21-880d-98b93b60bb7c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.626286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.627007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.627285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.627558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] PATH_INFO: `/v3/projects/f6bf62f144734ec1a011e4ab75dee4dc/users/cb98dc57a5dd44e29b1626406a1770b8/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.645078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6bf62f144734ec1a011e4ab75dee4dc, user_id=cb98dc57a5dd44e29b1626406a1770b8, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.647756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.659778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1198/2364] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:53:17 2022] PUT /identity/v3/projects/f6bf62f144734ec1a011e4ab75dee4dc/users/cb98dc57a5dd44e29b1626406a1770b8/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:53:17.669854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.670136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.670942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.698239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] MFA Rules not processed for user `cb98dc57a5dd44e29b1626406a1770b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:53:17.699467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:17.762358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1167/2365] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:53:17.767087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.826780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259ed1-0f86-4e5d-8876-d11292ae5f36 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.831351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.831944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.832221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.832487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.833102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.835025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.860605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1199/2366] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:53:17.864668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.876033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd58dcb-d721-4398-97ee-dba2babb1811 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.931861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.932461 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.932734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.933006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.933636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.935576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.956995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1168/2367] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:53:17 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:53:17.960984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:17.971460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c9381-0ae8-4b6d-9114-511f7c69dbf9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:17.975987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:17.976572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:17.976847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:17.977113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:17.977745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:17.979374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:17.990365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1200/2368] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:53:17.994500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:18.004486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c48daf-5323-4627-a579-50f37079362d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:18.009353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:18.009990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:18.010269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:18.010541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] PATH_INFO: `/v3/projects/b48ead90af964d8d907dcb3ea05d5375/users/888712e1096a40b78d5f42ed7d0b98ea/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:18.012792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:18.026912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48ead90af964d8d907dcb3ea05d5375, user_id=888712e1096a40b78d5f42ed7d0b98ea, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:18.029514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:18.042113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1169/2369] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:53:17 2022] PUT /identity/v3/projects/b48ead90af964d8d907dcb3ea05d5375/users/888712e1096a40b78d5f42ed7d0b98ea/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:53:18.046763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:18.056661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc2476-0eba-4a98-8c9d-b6360b3ee4f5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:18.075794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:18.113251 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:18.113856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:18.114136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:18.114411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:18.115084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:18.116692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09837660-5d28-4f75-8872-ef50fdca9760 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:18.126117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1201/2370] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:53:18.130435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:18.140788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1c320-7f17-4f78-8d64-0231389f26d2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:18.145512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:18.146197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:18.146476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:18.146775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] PATH_INFO: `/v3/projects/b48ead90af964d8d907dcb3ea05d5375/users/888712e1096a40b78d5f42ed7d0b98ea/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:18.148723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48ead90af964d8d907dcb3ea05d5375, user_id=888712e1096a40b78d5f42ed7d0b98ea, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:18.151455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:18.162124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1170/2371] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:53:18 2022] PUT /identity/v3/projects/b48ead90af964d8d907dcb3ea05d5375/users/888712e1096a40b78d5f42ed7d0b98ea/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:53:18.173869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:18.174355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:18.174464 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:18.201317 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] MFA Rules not processed for user `888712e1096a40b78d5f42ed7d0b98ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:53:18.202525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:18.268861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1202/2372] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:53:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:53:24.731740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:24.733567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:24.789670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469b0d2-64a9-4119-b32a-4303dabebd5a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:24.793240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:24.793825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:24.794099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:24.794366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:24.796071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:24.864380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:24.865798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:24.875131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1171/2373] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:53:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:53:24.920488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:24.930407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc2805-f33c-40ae-aa8f-c819570740f3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:24.934038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:24.934675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:24.934953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:24.935221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:24.937452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:25.006252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:25.007669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:25.017124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1203/2374] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:53:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:53:25.063306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:53:25.071800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db48a8b-3931-4d6c-a78b-953a9eee1587 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:53:25.075409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:53:25.075999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:53:25.076278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:53:25.077171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:53:25.078920 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:53:25.148607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:53:25.150026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:53:25.154455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:53:25.159759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1172/2375] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:53:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:53:35.012368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=85 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:11.025803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:54:11.025803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:54:11.037391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce584db3-43fc-4578-82c2-789acd820d3b service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:54:11.042169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:54:11.042835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:54:11.043248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:54:11.043523 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:54:11.045337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] Could not find project: cf5ba123a76540568cceaaa4fa54f269.: keystone.exception.ProjectNotFound: Could not find project: cf5ba123a76540568cceaaa4fa54f269. Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 18 06:54:11.066238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:54:11.067391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:54:11.068682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:54:11.069843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:54:11.071374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cf5ba123a76540568cceaaa4fa54f269. Sep 18 06:54:11.071374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:54:11.071374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1204/2376] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:54:11 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 44 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 18 06:54:11.236793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:54:11.245818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd972c-7d51-44d3-ba6f-7289071a9734 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:54:11.250142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:54:11.250791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:54:11.251070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:54:11.251343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:54:11.261228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:54:11.263153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:54:11.299606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1173/2377] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:54:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:54:21.049237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:21.268666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:54:53.690131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:54:53.692309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:54:53.762782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbc5ef-0ad6-4e65-bb3e-38b4d5fdddce service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:54:53.766413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:54:53.767052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:54:53.767332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:54:53.767602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:54:53.769327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:54:53.779287 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:54:53.780707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:54:53.788554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1205/2378] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:54:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:03.785005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=30 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:25.233603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:25.242749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b949120-d6f6-43c0-b35f-2f0777681a8d service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:25.246297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:25.246913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:25.247185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:25.247450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:25.249197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.305679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:25.307180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:25.315836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1174/2379] 10.222.0.20 () {60 vars in 1396 bytes} [Sun Sep 18 06:55:25 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:25.363347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:25.372373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21debe98-b0ba-46fd-a250-088591cf9850 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:25.376325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:25.376927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:25.377207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:25.377499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:25.412950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:25.415202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.442608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:25.443977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:25.454010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1206/2380] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:25.503195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857b168-f137-4e16-8113-f5bee9e44af8 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:25.509225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:25.509825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:25.510104 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:25.510371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:25.512151 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.570373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:25.572591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:25.582724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1175/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:55:25.621655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:25.634368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650d254-26d8-485c-8ba1-333620f8e918 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:25.640144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:25.641411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:25.641690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:25.642462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:25.644885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] Could not find project: 9cbc859294e9456eb2ca751279a17f52.: keystone.exception.ProjectNotFound: Could not find project: 9cbc859294e9456eb2ca751279a17f52. Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 18 06:55:25.659899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:55:25.661255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:55:25.662510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:55:25.663778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:55:25.664979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9cbc859294e9456eb2ca751279a17f52. Sep 18 06:55:25.664979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:55:25.664979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1207/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:25 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 18 06:55:25.855611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:25.857095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.871514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033245ae-46e5-499e-9ba8-274be811f4e8 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:25.877713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:25.878340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:25.878643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:25.878917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:25.880863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:25.914855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:25.943075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:25.944626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:25.954196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1176/2383] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:27.141885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:27.153577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281d638-b8c6-43db-892b-e0545533944f service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:27.159819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:27.160444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:27.160720 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:27.160990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:27.162759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-50915290-e9f6-4c67-b23e-b310a3c8896a service nova] Could not find project: d1e46b3e28a34fd7881d51a4c0bec2cf.: keystone.exception.ProjectNotFound: Could not find project: d1e46b3e28a34fd7881d51a4c0bec2cf. Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 18 06:55:27.174826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:55:27.176046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 18 06:55:27.177286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 18 06:55:27.178363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 18 06:55:27.179441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d1e46b3e28a34fd7881d51a4c0bec2cf. Sep 18 06:55:27.179441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Sep 18 06:55:27.179441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1208/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:27 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 18 06:55:28.023669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:28.034401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d51c4-bfdd-4ca8-acc2-1ff4a0518a00 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:28.049268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:28.049894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:28.050219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:28.050503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:28.052317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:28.124189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:28.125915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:28.136216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1177/2385] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:30.276111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:30.276669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:30.276911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:30.310570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] MFA Rules not processed for user `d85160b01d824bc3a32b425fab98d2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:55:30.312603 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:30.319764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1209/2386] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:55:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:55:30.804901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:30.816031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb869241-8ab5-4a9a-b180-0e6b1a7a70b9 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:30.820254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:30.820923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:30.821281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:30.821735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:30.823506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:30.833293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:30.834726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:30.845988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1178/2387] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:31.668903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:31.671071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:31.764233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c9bf7-70c3-4680-b86e-b8a710171d0a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:31.784793 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:31.785685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:31.786180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:31.787332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:31.790094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:31.809069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:31.810726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:31.829633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1210/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:55:32.341717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:32.343456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:32.360939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc245cc6-a59c-4cba-bd4f-2385e8c1de5c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:32.366422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:32.367097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:32.367375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:32.367645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:32.369402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:32.382249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:32.384792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:32.402074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1179/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:35.747155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:35.764385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed3e5-d2ba-423e-866f-ddd6f547daae service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:35.771461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:35.772804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:35.773346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:35.773747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:35.775813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:35.790828 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:35.792569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:35.805260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1211/2390] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:37.138360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:37.139915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:37.153754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9944d78-c328-4434-a846-19f1a5a739a5 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:37.158852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:37.159470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:37.159750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:37.160022 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] PATH_INFO: `/v3/users/c2b511ca42e6427aa2f1627c87ad3fad` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:37.161131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b511ca42e6427aa2f1627c87ad3fad)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:37.163077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:37.208496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b511ca42e6427aa2f1627c87ad3fad'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.215449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b511ca42e6427aa2f1627c87ad3fad'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.216050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b511ca42e6427aa2f1627c87ad3fad'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.216644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b511ca42e6427aa2f1627c87ad3fad'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.224570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b511ca42e6427aa2f1627c87ad3fad'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.242850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1180/2391] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:55:37 2022] DELETE /identity/v3/users/c2b511ca42e6427aa2f1627c87ad3fad => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:55:37.473031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:37.486891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe94f07-bd7d-4ce3-844c-f7a14e69805b service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:37.529946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:37.544968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:37.545638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:37.545994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:37.546329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] PATH_INFO: `/v3/projects/d0adbb8b987449f08ee39d325424b1da` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:37.547466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0adbb8b987449f08ee39d325424b1da)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:37.549365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:37.596093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0adbb8b987449f08ee39d325424b1da', 'request_id': 'req-dd6c0b68-4cff-47e9-8417-a8861851f15c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.602202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0adbb8b987449f08ee39d325424b1da', 'request_id': 'req-dd6c0b68-4cff-47e9-8417-a8861851f15c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.610868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0adbb8b987449f08ee39d325424b1da', 'request_id': 'req-dd6c0b68-4cff-47e9-8417-a8861851f15c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.621331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] The token cache is being invalidate because project d0adbb8b987449f08ee39d325424b1da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:55:37.621618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.622243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.623748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1212/2392] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:55:37 2022] DELETE /identity/v3/projects/d0adbb8b987449f08ee39d325424b1da => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:55:37.627804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:37.629539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:37.644363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dca9e9-5820-4626-b165-6cd0fb610a09 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:37.655143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:37.706430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:37.707089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:37.707373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:37.707647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] PATH_INFO: `/v3/users/dcab7516ea6640d1a4a8e3231c518ea7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:37.721657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcab7516ea6640d1a4a8e3231c518ea7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:37.723639 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:37.773278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcab7516ea6640d1a4a8e3231c518ea7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.781062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcab7516ea6640d1a4a8e3231c518ea7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.782136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcab7516ea6640d1a4a8e3231c518ea7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.782791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcab7516ea6640d1a4a8e3231c518ea7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.790474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcab7516ea6640d1a4a8e3231c518ea7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:37.807990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1181/2393] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:55:37 2022] DELETE /identity/v3/users/dcab7516ea6640d1a4a8e3231c518ea7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:55:37.988661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:37.990129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:38.017737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c0b68-4cff-47e9-8417-a8861851f15c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:38.078715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:38.079447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:38.079751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:38.080023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] PATH_INFO: `/v3/projects/a4df4f13709a46a5a5891e28798c8b19` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:38.090554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4df4f13709a46a5a5891e28798c8b19)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:38.092517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:38.154063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4df4f13709a46a5a5891e28798c8b19', 'request_id': 'req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:38.160815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4df4f13709a46a5a5891e28798c8b19', 'request_id': 'req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:38.170499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4df4f13709a46a5a5891e28798c8b19', 'request_id': 'req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:38.176389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:38.184945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] The token cache is being invalidate because project a4df4f13709a46a5a5891e28798c8b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:55:38.185669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:38.186508 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:55:38.189096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1213/2394] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:55:37 2022] DELETE /identity/v3/projects/a4df4f13709a46a5a5891e28798c8b19 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:55:40.923550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:55:40.927883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:41.003320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6068e-064b-49f5-be88-caa2c7f68bb7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:55:41.019963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:41.067958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:55:41.068557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:55:41.068835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:55:41.069107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:55:41.070879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:55:41.082289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:55:41.083843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae4d45-9313-47b4-bb21-7e2994c748c7 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:55:41.094210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1182/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:55:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:55:48.189734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:55:51.089753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=25 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:10.421645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:57:10.423091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:10.490075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828713-af9a-4be6-9b0b-26a4cbf7f4d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:57:10.493685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:57:10.494285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:57:10.494562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:57:10.494864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:57:10.496563 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:10.516743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:57:10.518170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:57:10.538929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1214/2396] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:57:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:57:20.536426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=38 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:24.828349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:57:24.828907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:57:24.828907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:57:24.866849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] MFA Rules not processed for user `d85160b01d824bc3a32b425fab98d2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:57:24.868099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:24.933983 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1183/2397] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:57:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:57:25.776108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:57:25.777514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:25.878976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fabaff-68ab-4a50-9468-83f9b727d80c service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:57:25.883042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:57:25.883697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:57:25.884117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:57:25.884519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:57:25.888023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:25.905514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:57:25.907329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:57:25.919500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1215/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:57:34.931147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=37 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:35.915009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=39 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:57:58.980442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:57:58.981922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:59.067769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9512668-9153-4ed1-9c43-109b3d7bbe3d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:57:59.072705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:57:59.073300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:57:59.073577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:57:59.073846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:57:59.076141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:57:59.087304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:57:59.088864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:57:59.099260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1184/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:57:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:58:09.095083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=37 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:09.801367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:09.802802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:09.817166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b56260-7be7-43fc-9155-da6a76d1abd1 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:09.822688 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:09.823279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:09.823560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:09.823831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:09.825483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:09.835588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:09.837615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:09.846442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1216/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:19.842412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:21.004008 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:21.005336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:21.017102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c77aa-3ec2-4274-8f17-43944dee6538 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:21.020862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:21.021479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:21.021751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:21.022020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:21.024258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:21.099012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:21.101040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:21.120807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1185/2401] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:21.153673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:21.167670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca797a-84be-4b55-b499-f7e991e81d18 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:21.172801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:21.173923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:21.174185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:21.174727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:21.175242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:21.177799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:21.183195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bfeb4-2b3d-4a9c-9037-47ee92b79392 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:21.187586 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:21.188189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:21.188466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:21.188759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:21.190518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:21.262503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:21.264020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:21.276127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1217/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:21.289119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:21.291267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff7d67-8f3f-4f35-9606-cac80d9e96f0 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:21.312202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1186/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:21.864319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:21.866331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:21.951590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5116f-4add-4866-8052-d634cfc949b5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:21.959451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:21.960531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:21.961042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:21.962064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] PATH_INFO: `/v3/users/cb98dc57a5dd44e29b1626406a1770b8` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:21.964058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb98dc57a5dd44e29b1626406a1770b8)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:21.967900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:22.020382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb98dc57a5dd44e29b1626406a1770b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.028767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb98dc57a5dd44e29b1626406a1770b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.029490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb98dc57a5dd44e29b1626406a1770b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.030227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb98dc57a5dd44e29b1626406a1770b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.043023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb98dc57a5dd44e29b1626406a1770b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.055567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1218/2404] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:21 2022] DELETE /identity/v3/users/cb98dc57a5dd44e29b1626406a1770b8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:22.060180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:22.060684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:22.060822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:22.079705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:22.080948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:22.139109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1187/2405] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:22.149301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:22.150774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:22.165418 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:22.209958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a27563-4901-43b5-b577-61e90bc463ed admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:22.214183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:22.214823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:22.215105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:22.215373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:22.224572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:22.225998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:22.235743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1219/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:22.591089 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:22.592642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:22.609659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:22.616101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adb434-89a6-40fa-a522-821e51dfdbd1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:22.623889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:22.625446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:22.627644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:22.628169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] PATH_INFO: `/v3/projects/f6bf62f144734ec1a011e4ab75dee4dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:22.629585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6bf62f144734ec1a011e4ab75dee4dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:22.631939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:22.717939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6bf62f144734ec1a011e4ab75dee4dc', 'request_id': 'req-b15b6b04-5b58-4900-bbec-8e0834b5381d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.727696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6bf62f144734ec1a011e4ab75dee4dc', 'request_id': 'req-b15b6b04-5b58-4900-bbec-8e0834b5381d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.743123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6bf62f144734ec1a011e4ab75dee4dc', 'request_id': 'req-b15b6b04-5b58-4900-bbec-8e0834b5381d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.770094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] The token cache is being invalidate because project f6bf62f144734ec1a011e4ab75dee4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:22.770806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.771921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.777844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1188/2407] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:22 2022] DELETE /identity/v3/projects/f6bf62f144734ec1a011e4ab75dee4dc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:22.779162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:22.792640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da2273-4880-46a3-ac60-26e0282aa9de service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:22.867144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:22.867947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:22.868230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:22.868511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] PATH_INFO: `/v3/users/888712e1096a40b78d5f42ed7d0b98ea` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:22.869782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888712e1096a40b78d5f42ed7d0b98ea)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:22.871979 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:22.940200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888712e1096a40b78d5f42ed7d0b98ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.951113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888712e1096a40b78d5f42ed7d0b98ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.951990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888712e1096a40b78d5f42ed7d0b98ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.952831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888712e1096a40b78d5f42ed7d0b98ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.968001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888712e1096a40b78d5f42ed7d0b98ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:22.993258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1220/2408] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:22 2022] DELETE /identity/v3/users/888712e1096a40b78d5f42ed7d0b98ea => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:23.248928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:23.252303 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:23.452915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6b04-5b58-4900-bbec-8e0834b5381d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:23.560736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:23.563062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:23.607914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:23.608701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.610123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.610521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.617936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:23.635308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:23.646416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:23.648317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55855811-2e89-487f-af1d-bb91d92f9834 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:23.670568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e07c4-5e6c-482c-abed-a94aebeb3054 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:23.672077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1189/2409] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:23.677541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:23.678190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.678475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.678794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] PATH_INFO: `/v3/projects/b48ead90af964d8d907dcb3ea05d5375` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.680491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48ead90af964d8d907dcb3ea05d5375)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:23.682616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:23.738041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48ead90af964d8d907dcb3ea05d5375', 'request_id': 'req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:23.745308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48ead90af964d8d907dcb3ea05d5375', 'request_id': 'req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:23.754211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48ead90af964d8d907dcb3ea05d5375', 'request_id': 'req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:23.765126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] The token cache is being invalidate because project b48ead90af964d8d907dcb3ea05d5375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:23.765419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:23.766049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fb34720e-22f4-41fd-9f10-01a2ef1ff0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:23.767567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1221/2410] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:23 2022] DELETE /identity/v3/projects/b48ead90af964d8d907dcb3ea05d5375 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:23.773320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.773654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.773929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.795126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:23.796368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:23.800627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:23.867356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1190/2411] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:23.875434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.876121 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.876481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.904712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:23.906133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:23.916067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1222/2412] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:23.920446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:23.931701 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a3033-cab2-4c59-ba54-56d5e02091a0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:23.936722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:23.937407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.937923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.938392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.940494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:23.941006 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:23.942734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:23.957973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1191/2413] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:59:23.962847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:23.974472 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb836f17-3c5b-4209-9f88-8ef04fa680cc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:23.982894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:23.983642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:23.984013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:23.984426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:23.985256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:23.987777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.043993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1223/2414] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:23 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:24.049023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.068205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e817a-2df1-46a3-84e3-0454160248a9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.143679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.144538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.145003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.145346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.146147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.149228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.176958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1192/2415] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:24 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:24.181133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.191668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeadf4b-d576-4f39-ae19-7a7227f62819 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.196998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.197644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.197923 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.198222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.198912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.200544 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.212286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1224/2416] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:24.215986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.226479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6814fb8-b29b-42e2-a4e6-74c974326f16 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.231939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.232556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.232866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.233264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.233929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.235582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337c6e5-cc55-47bd-a200-8f4cd0a0d686 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.246831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1193/2417] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:24.251297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.261695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b29c88-2885-4885-aa12-b63387492171 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.266654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.267311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.267588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.267861 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] PATH_INFO: `/v3/projects/30313fb1f70d4e50b1d628c5be903fa5/users/5f64c48c0c8d4724956d0005d6d32f06/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.285352 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30313fb1f70d4e50b1d628c5be903fa5, user_id=5f64c48c0c8d4724956d0005d6d32f06, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.288165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.303091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1225/2418] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:24 2022] PUT /identity/v3/projects/30313fb1f70d4e50b1d628c5be903fa5/users/5f64c48c0c8d4724956d0005d6d32f06/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:24.315019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.315552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.315552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.344760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] MFA Rules not processed for user `5f64c48c0c8d4724956d0005d6d32f06`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:24.345990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:24.422633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1194/2419] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:24.434050 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.436191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:24.511864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb395b-44cd-40c7-8159-630e2dddbba6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.515898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.516509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.516792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.517068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.527269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.529195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dcdba5-2686-4d24-88bf-debecc87ec77 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.540742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1226/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:24.562900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.568559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:24.637256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22995a9-af1d-4d49-bd93-5a3b0aa6b1ec None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.641939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.642569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.642878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.643154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] PATH_INFO: `/v3/users/5f64c48c0c8d4724956d0005d6d32f06` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.644295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f64c48c0c8d4724956d0005d6d32f06)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.646200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.699194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f64c48c0c8d4724956d0005d6d32f06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:24.706980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f64c48c0c8d4724956d0005d6d32f06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:24.707628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f64c48c0c8d4724956d0005d6d32f06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:24.708261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f64c48c0c8d4724956d0005d6d32f06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:24.715497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f64c48c0c8d4724956d0005d6d32f06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:24.734508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1195/2421] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:24 2022] DELETE /identity/v3/users/5f64c48c0c8d4724956d0005d6d32f06 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:24.738839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.739308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.739406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.747916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:24.761433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:24.763024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:24.822765 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1227/2422] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:24.832504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:24.833907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:24.890549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af45ac2-44d3-4ffd-a75d-3409564f05f1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:24.894247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:24.894867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:24.895146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:24.895413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:24.904651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:24.906070 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4682a-c444-4369-81c4-96fb71527f56 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:24.916214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1196/2423] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:25.190841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.192931 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:25.210202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95faa04-5d3e-4cf5-b85a-34fe659bffbb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.220633 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.221460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.221707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.221977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] PATH_INFO: `/v3/projects/30313fb1f70d4e50b1d628c5be903fa5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.223865 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30313fb1f70d4e50b1d628c5be903fa5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.226658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.313792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30313fb1f70d4e50b1d628c5be903fa5', 'request_id': 'req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:25.332069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30313fb1f70d4e50b1d628c5be903fa5', 'request_id': 'req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:25.344609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30313fb1f70d4e50b1d628c5be903fa5', 'request_id': 'req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:25.361190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] The token cache is being invalidate because project 30313fb1f70d4e50b1d628c5be903fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:25.361492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:25.362157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b13e7ac0-cbe7-4479-88f3-cb608ebb982f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:25.363913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1228/2424] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:25 2022] DELETE /identity/v3/projects/30313fb1f70d4e50b1d628c5be903fa5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:25.369402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.370003 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.370212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.395575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:25.396808 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:25.420590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:25.470090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1197/2425] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:25.478109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.478602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.478977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.504491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:25.505790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:25.516384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1229/2426] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:25.520315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.531479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdbc08-11b8-4633-aa00-7c1e483e0345 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.536307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.536909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.537189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.537462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.538106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:25.538408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.540267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.559058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1198/2427] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:59:25.559663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.569041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3e9da-0cf1-4c39-93e1-e29c814e6306 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.575163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.575714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.575989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.576255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.577044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.578903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.616238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1230/2428] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:25 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:25.621839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.634043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f609061-3e1b-4997-9042-8e5f7a1e3113 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.730086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.731032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.731525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.732119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.733156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.735550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.764881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1199/2429] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:25 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:25.766148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.776946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc149f-503d-4e25-b1f7-4abd17cac56d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.782145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.782783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.783065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.783344 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.784022 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.785749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.808179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1231/2430] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:25.812587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.834225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f8c14-85be-4dbd-881d-effbcca322bf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.848629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.849556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.849885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.850161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.850842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.852963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c04b06-7888-4814-9cd3-3f5352d9707a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.885212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1200/2431] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:25.890534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:25.901218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5acd3-619f-4f52-910d-0a0212450829 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:25.905943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:25.906605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.906898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.907294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] PATH_INFO: `/v3/projects/ab7e6b71a22744b78f491298d55b4310/users/21e0e5c1ef514053b3a7fb0f7409e568/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.927794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7e6b71a22744b78f491298d55b4310, user_id=21e0e5c1ef514053b3a7fb0f7409e568, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:25.931196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:25.943843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1232/2432] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:25 2022] PUT /identity/v3/projects/ab7e6b71a22744b78f491298d55b4310/users/21e0e5c1ef514053b3a7fb0f7409e568/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:25.955885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:25.956359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:25.956729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:25.988100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] MFA Rules not processed for user `21e0e5c1ef514053b3a7fb0f7409e568`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:25.989366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:26.054368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1201/2433] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:26.066220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:26.067955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:26.096422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:26.151588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f9774-0e94-45ca-aded-491b524cd183 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:26.155497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:26.156105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:26.156388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:26.156661 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:26.164973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:26.166406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:26.175266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1233/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:36.051091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:36.171218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=24 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:38.363400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:38.364803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:38.422347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56760349-0275-4e05-95fc-d1d943e53b84 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:38.426001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:38.426624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:38.426906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:38.427178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:38.428912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:38.511799 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:38.513284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:38.525544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1202/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:41.515211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:41.517735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:41.545152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496028ed-dffd-4869-a593-b74f3a1bebec service glance] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:41.550795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:41.551337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:41.551616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:41.551890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] PATH_INFO: `/v3/users/7a6b8fbd31464e4d99449c74b4d1dda5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:41.553186 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6b8fbd31464e4d99449c74b4d1dda5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:41.555153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:41.611833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6b8fbd31464e4d99449c74b4d1dda5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:41.632567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6b8fbd31464e4d99449c74b4d1dda5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:41.633322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6b8fbd31464e4d99449c74b4d1dda5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:41.633965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6b8fbd31464e4d99449c74b4d1dda5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:41.653572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aab967f5-c114-49d7-8bfe-4d47be23354a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6b8fbd31464e4d99449c74b4d1dda5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:41.670147 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1234/2436] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:41 2022] DELETE /identity/v3/users/7a6b8fbd31464e4d99449c74b4d1dda5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:41.900838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:41.911285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3680779-3226-46f0-b501-8602f02b597e service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:41.973107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:41.973894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:41.974193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:41.974469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] PATH_INFO: `/v3/projects/91ef3478fd574180abe99393ba769124` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:41.975697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ef3478fd574180abe99393ba769124)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:41.977622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.032146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ef3478fd574180abe99393ba769124', 'request_id': 'req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:42.034014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:42.038736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ef3478fd574180abe99393ba769124', 'request_id': 'req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:42.047284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ef3478fd574180abe99393ba769124', 'request_id': 'req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:42.057401 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] The token cache is being invalidate because project 91ef3478fd574180abe99393ba769124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:42.057687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:42.058306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3cc911cd-9fd1-4095-a2ee-1ac8f350f5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:42.059796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1203/2437] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:41 2022] DELETE /identity/v3/projects/91ef3478fd574180abe99393ba769124 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:42.065991 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.066275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.066547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.085730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:42.086982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:42.144215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1235/2438] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:42.152360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.152738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.153038 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.173240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:42.174450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:42.182391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1204/2439] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:42.186425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.195882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebda90-d85a-482c-a7c0-d2827a468942 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.200768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.201355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.201631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.201900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.202543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:42.202875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.204455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.216153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1236/2440] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:59:42.220252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.230286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478727a-5fd4-41c8-9069-a9836165df58 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.235069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.235657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.235934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.236202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.236833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.238777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.265999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1205/2441] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:42.270089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.281383 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf702d-a007-4d91-9ace-abc9d16a2556 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.336466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.337597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.337909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.338206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.338911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.340862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.348743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:42.362786 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1237/2442] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:42.366856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.377480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af908eeb-20ae-4275-9c97-fbd16b8187d5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.382047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.382665 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.382942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.383210 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.383869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.385940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.395770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1206/2443] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:42.399682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.410097 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada06136-b8fd-47a6-8056-0b707b7b0a9c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.414742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.415337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.415617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.415895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.416535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.418146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ba5b2-4ddf-43d5-a29f-a16054e00d2c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.429053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1238/2444] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:42.433116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.443169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3631ffab-614d-4182-ad66-5d68746354f1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.448232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.449023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.449188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.449443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] PATH_INFO: `/v3/projects/0bff1d6fde5d4ea5937f8d2110566f73/users/50b3b7fa1a4549b5a78c650e3f2c1c65/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.469612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bff1d6fde5d4ea5937f8d2110566f73, user_id=50b3b7fa1a4549b5a78c650e3f2c1c65, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.472387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.485591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1207/2445] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:42 2022] PUT /identity/v3/projects/0bff1d6fde5d4ea5937f8d2110566f73/users/50b3b7fa1a4549b5a78c650e3f2c1c65/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:42.496181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.496467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.496741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.525586 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] MFA Rules not processed for user `50b3b7fa1a4549b5a78c650e3f2c1c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:42.526827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:42.589290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1239/2446] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:42.593854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.654834 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a00ef3-0599-472d-97d6-5db725d2ec08 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.659490 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.660082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.660378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.660635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.661267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.663212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.689250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1208/2447] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:42.698767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.709322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fb8a1-ed4d-47ad-8729-eeb8a7d07a3b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.767475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.768084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.768392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.768666 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.769308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.771326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.792687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1240/2448] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:42 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:42.796559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.807887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1687b-5ecf-407f-b00b-f678fdd862bb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.812615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.813215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.813492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.813784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.814416 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.816037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.827517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1209/2449] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:42.831555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.843906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12789365-9aec-4718-bc5a-41328459db80 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.849606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.850294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.850625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.850955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] PATH_INFO: `/v3/projects/8dcf0446245e43b58403f07994e4350a/users/5c554053fba44a2ca708efb25a97ff08/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.866274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcf0446245e43b58403f07994e4350a, user_id=5c554053fba44a2ca708efb25a97ff08, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.869570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.888030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1241/2450] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:42 2022] PUT /identity/v3/projects/8dcf0446245e43b58403f07994e4350a/users/5c554053fba44a2ca708efb25a97ff08/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:42.892386 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:42.910376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e926fd-2a36-484f-9cee-e26c787b111c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:42.960596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:42.972072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:42.972830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:42.973392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:42.973604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:42.974248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:42.975862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f462e72-dc18-46e9-9687-5ead7cc9d3c4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:42.986902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1210/2451] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:42.990962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:43.001584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176f1d-d56e-42f6-9be5-78da4014f748 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:43.006116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:43.006818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.007092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.007360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] PATH_INFO: `/v3/projects/8dcf0446245e43b58403f07994e4350a/users/5c554053fba44a2ca708efb25a97ff08/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.009327 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcf0446245e43b58403f07994e4350a, user_id=5c554053fba44a2ca708efb25a97ff08, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:43.011972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:43.022193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1242/2452] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:42 2022] PUT /identity/v3/projects/8dcf0446245e43b58403f07994e4350a/users/5c554053fba44a2ca708efb25a97ff08/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:43.034821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.035185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.035570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.066111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] MFA Rules not processed for user `5c554053fba44a2ca708efb25a97ff08`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:43.067633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:43.136453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1211/2453] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:43.147274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:43.148973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:43.171550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:43.206309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ada8c-60b8-467b-8aa4-ec5d9cc2acaa admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:43.209960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:43.210555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.210863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.211131 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.220552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:43.221978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421f6ef-0da1-45cd-b6cf-dd6a4b99fd9c service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:43.231470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1243/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:43 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:43.375017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:43.440948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012759d-bca9-4d7d-8559-0b9051caf8f1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:43.445378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:43.445998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.446280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.447078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] PATH_INFO: `/v3/users/50b3b7fa1a4549b5a78c650e3f2c1c65` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.448188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b3b7fa1a4549b5a78c650e3f2c1c65)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:43.450147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:43.503602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b3b7fa1a4549b5a78c650e3f2c1c65'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:43.519306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b3b7fa1a4549b5a78c650e3f2c1c65'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:43.519635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b3b7fa1a4549b5a78c650e3f2c1c65'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:43.520380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b3b7fa1a4549b5a78c650e3f2c1c65'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:43.527493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b3b7fa1a4549b5a78c650e3f2c1c65'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:43.535093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:43.546047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1212/2455] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:43 2022] DELETE /identity/v3/users/50b3b7fa1a4549b5a78c650e3f2c1c65 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:43.550419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.550924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.551049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.569860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:43.571098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:43.635357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1244/2456] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:43.645611 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:43.647097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:43.717404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61734fd-0f8f-44f2-802e-c0b4b815c729 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:43.720982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:43.721590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:43.721865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:43.722135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:43.731588 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:43.733013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:43.743891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1213/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:44.015841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.017364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:44.030922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f1ba4-8ff3-4fb6-b685-9fcd36b4ed55 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:44.035938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:44.036610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.036891 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.037164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] PATH_INFO: `/v3/projects/0bff1d6fde5d4ea5937f8d2110566f73` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.038358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bff1d6fde5d4ea5937f8d2110566f73)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:44.040879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:44.089846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bff1d6fde5d4ea5937f8d2110566f73', 'request_id': 'req-7f01816e-5c56-43d8-8a16-23c039a8634d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.098368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bff1d6fde5d4ea5937f8d2110566f73', 'request_id': 'req-7f01816e-5c56-43d8-8a16-23c039a8634d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.108065 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bff1d6fde5d4ea5937f8d2110566f73', 'request_id': 'req-7f01816e-5c56-43d8-8a16-23c039a8634d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.119382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] The token cache is being invalidate because project 0bff1d6fde5d4ea5937f8d2110566f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:44.119675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.120310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.123156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1245/2458] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:44 2022] DELETE /identity/v3/projects/0bff1d6fde5d4ea5937f8d2110566f73 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:44.126161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.135917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976afa3d-a203-4210-be25-ce1fbb1464e5 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:44.193604 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:44.194257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.194532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.194835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] PATH_INFO: `/v3/users/5c554053fba44a2ca708efb25a97ff08` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.195982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c554053fba44a2ca708efb25a97ff08)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:44.197905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:44.245641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c554053fba44a2ca708efb25a97ff08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.252335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c554053fba44a2ca708efb25a97ff08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.253027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c554053fba44a2ca708efb25a97ff08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.253628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c554053fba44a2ca708efb25a97ff08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.261157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1bde189c-36b9-4637-abbb-e284d9c597ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c554053fba44a2ca708efb25a97ff08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.278299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1214/2459] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:44 2022] DELETE /identity/v3/users/5c554053fba44a2ca708efb25a97ff08 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:44.569967 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.583795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:44.598350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01816e-5c56-43d8-8a16-23c039a8634d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:44.641626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:44.656665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:44.657299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.657579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.657849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] PATH_INFO: `/v3/projects/8dcf0446245e43b58403f07994e4350a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.658992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dcf0446245e43b58403f07994e4350a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:44.660906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:44.705490 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dcf0446245e43b58403f07994e4350a', 'request_id': 'req-7af8cb61-c8b5-4422-8fdc-704ed055980b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.711501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dcf0446245e43b58403f07994e4350a', 'request_id': 'req-7af8cb61-c8b5-4422-8fdc-704ed055980b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.719294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dcf0446245e43b58403f07994e4350a', 'request_id': 'req-7af8cb61-c8b5-4422-8fdc-704ed055980b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.729510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] The token cache is being invalidate because project 8dcf0446245e43b58403f07994e4350a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:44.729797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.730404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7af8cb61-c8b5-4422-8fdc-704ed055980b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:44.731899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1246/2460] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:44 2022] DELETE /identity/v3/projects/8dcf0446245e43b58403f07994e4350a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:44.736746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.737029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.737328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.756541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:44.757732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:44.766318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:44.817933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1215/2461] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:44.826332 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.826665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.826947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.849031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:44.850279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:44.859117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1247/2462] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:44.863494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.873886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565abbbb-fc8f-4aad-931e-4ce575259bef None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:44.879217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:44.879849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.880128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.880451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.881223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:44.881613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:44.883300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:44.900456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1216/2463] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:59:44.902013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.911910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a2a2d-4b54-429e-96ba-a956cf84a50a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:44.918594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:44.919326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:44.919625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:44.919880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:44.920537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:44.922485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:44.952616 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1248/2464] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:44 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:44.958836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:44.969774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49506a38-a09b-4225-9824-4e39ebffaa85 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:45.031188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:45.031870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.032231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.032598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.033272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:45.035275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:45.055504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1217/2465] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:44 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:45.060263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:45.070248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9802f-3ff5-431b-b171-222403716d9f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:45.075961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:45.076580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.076852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.077116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.077752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:45.080198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:45.093390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1249/2466] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:45.097534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:45.107680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f944b-de95-4140-a91a-dfb946ef514e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:45.112382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:45.112973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.113253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.113528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.114172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:45.115877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0715ac-518f-40f1-97ad-f662b84fff2a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:45.130251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1218/2467] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:45.135099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:45.145201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c272257-8199-4788-a4f2-f1ef877dff88 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:45.149773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:45.150443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.150754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.151029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] PATH_INFO: `/v3/projects/a68067e070dd4da380f6378faba35193/users/69ab1bc171834c87bd0614050d92a7e9/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.285716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68067e070dd4da380f6378faba35193, user_id=69ab1bc171834c87bd0614050d92a7e9, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:45.288838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:45.303280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1250/2468] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:45 2022] PUT /identity/v3/projects/a68067e070dd4da380f6378faba35193/users/69ab1bc171834c87bd0614050d92a7e9/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:45.308728 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cae12-c7d6-4e5d-a71f-026544a7d257 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.309066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cae12-c7d6-4e5d-a71f-026544a7d257 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.309342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cae12-c7d6-4e5d-a71f-026544a7d257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.330923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4b1cae12-c7d6-4e5d-a71f-026544a7d257 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:45.332139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b1cae12-c7d6-4e5d-a71f-026544a7d257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:45.392895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1219/2469] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:45.402546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:45.404250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:45.465208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba0316-67a6-4e49-b88e-7f6a12bffea7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:45.469035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:45.469645 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:45.469917 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:45.470181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:45.480526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:45.481977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:45.491537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1251/2470] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:46.181380 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.181948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.182252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.214853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] MFA Rules not processed for user `69ab1bc171834c87bd0614050d92a7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:46.216189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:46.225651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:46.291634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1220/2471] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:46.296211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.305832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ba76c-4b91-49db-afec-e133ca1a3c4f service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.310378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.310999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.311275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.311542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.312162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.314113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.329129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:46.345599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1252/2472] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:46 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:46.349625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.360500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e00335-0bcf-4f8f-aac1-7856c0c7f26b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.424261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.424871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.425148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.425417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.426047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.428300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.453777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1221/2473] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:46 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:46.458560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.473179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24107b-92b5-411f-8bc5-babb58f31b53 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.478452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.479079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.479379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.479647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.480467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.482066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.495502 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1253/2474] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:46.499777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.509955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069e8dd-ba59-41b2-9c27-6897d51c581b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.514507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.515170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.515444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.515714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] PATH_INFO: `/v3/projects/9372029bff8349249685682d58a87a15/users/3367ae9ff0af477fb672ae4caa5fae77/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.530546 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9372029bff8349249685682d58a87a15, user_id=3367ae9ff0af477fb672ae4caa5fae77, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.533251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.546175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1222/2475] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:46 2022] PUT /identity/v3/projects/9372029bff8349249685682d58a87a15/users/3367ae9ff0af477fb672ae4caa5fae77/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:46.550271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.560361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bf3f4-134a-4ac5-8834-bc6a27cbc676 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.617438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.617972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.618260 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.618526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.619200 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.620853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.635555 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1254/2476] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:46.636479 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:46.646596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d257b-e3d9-4712-ac99-b5b5692ed096 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:46.651944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:46.652590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:46.652868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:46.653135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] PATH_INFO: `/v3/projects/9372029bff8349249685682d58a87a15/users/3367ae9ff0af477fb672ae4caa5fae77/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:46.655121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9372029bff8349249685682d58a87a15, user_id=3367ae9ff0af477fb672ae4caa5fae77, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:46.657809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b8a17-1871-4edb-8adf-666e4b328f40 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:46.668533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1223/2477] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:46 2022] PUT /identity/v3/projects/9372029bff8349249685682d58a87a15/users/3367ae9ff0af477fb672ae4caa5fae77/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:47.095161 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:47.097757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:47.132842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c622ada-b05e-469e-85cf-d5ab735d9f10 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:47.220039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:47.220677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.220951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.221222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] PATH_INFO: `/v3/users/21e0e5c1ef514053b3a7fb0f7409e568` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:47.222348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e0e5c1ef514053b3a7fb0f7409e568)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:47.224315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:47.273244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e0e5c1ef514053b3a7fb0f7409e568'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.280820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e0e5c1ef514053b3a7fb0f7409e568'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.281444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e0e5c1ef514053b3a7fb0f7409e568'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.282127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e0e5c1ef514053b3a7fb0f7409e568'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.295480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-99d6cb20-12c0-4c5a-b38d-0ba56cfdd021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e0e5c1ef514053b3a7fb0f7409e568'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.303807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:47.311635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1255/2478] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:47 2022] DELETE /identity/v3/users/21e0e5c1ef514053b3a7fb0f7409e568 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:47.316139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.316653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.316781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:47.336695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:47.337950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:47.422787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1224/2479] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:47.424331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f989c2-681b-4676-890f-43f426e345af None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.424815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f989c2-681b-4676-890f-43f426e345af None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.425183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f989c2-681b-4676-890f-43f426e345af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:47.433111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:47.434765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:47.443198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:47.470611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-51f989c2-681b-4676-890f-43f426e345af None None] MFA Rules not processed for user `3367ae9ff0af477fb672ae4caa5fae77`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:47.471966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-51f989c2-681b-4676-890f-43f426e345af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:47.522459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b92e5-7ce8-4f4d-80c0-6253676e96dc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:47.526813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:47.527419 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.527692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.527971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:47.540269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:47.541828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:47.558952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1225/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:47.580674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1256/2481] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:47 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:47.874628 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:47.876517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:47.894762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b50aff-ff6b-4d46-91cc-aa61b6a7fd4d service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:47.900472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:47.901181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.901530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.901905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] PATH_INFO: `/v3/projects/ab7e6b71a22744b78f491298d55b4310` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:47.903261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7e6b71a22744b78f491298d55b4310)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:47.905792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:47.954789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7e6b71a22744b78f491298d55b4310', 'request_id': 'req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.960907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7e6b71a22744b78f491298d55b4310', 'request_id': 'req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.969410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7e6b71a22744b78f491298d55b4310', 'request_id': 'req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.981800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] The token cache is being invalidate because project ab7e6b71a22744b78f491298d55b4310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:47.982087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.982880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7d614b2-9fdf-437b-ab14-1a1e7ae89d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:47.984433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1226/2482] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:47 2022] DELETE /identity/v3/projects/ab7e6b71a22744b78f491298d55b4310 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:47.989289 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:47.989602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:47.989893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.009777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:48.011048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:48.078846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1257/2483] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:48.088528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.089462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.089809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.113354 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:48.114705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:48.122904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1227/2484] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:48.127785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.137888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58411a50-1538-4d06-94f0-784a66dfbeba None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.142727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.143356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.143634 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.143905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.144571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:48.144876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.146511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.160818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1258/2485] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:59:48.165819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.183072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0d60c-47a7-420c-b5a8-f98845da4264 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.187475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.188110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.188391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.188664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.189312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.191315 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.223253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1228/2486] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:48 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:48.228247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.242082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe710c5-c41f-40c1-b978-2b32dfd8c25c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.305416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.306750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.307162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.307598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.308276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.310265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.327920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:48.334751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1259/2487] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:48 2022] POST /identity/v3/users => generated 280 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:48.338923 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.350861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5e937-2804-45ed-8e36-49473a90f758 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.358747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.360189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.360583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.360921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.362248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.364545 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.380836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1229/2488] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:48.385238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.396233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c2ed7-5bb7-472f-b341-076b7d92f162 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.401687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.402362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.402742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.403085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.403796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.405441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c16cdd-6c1f-4003-9a65-43926b6e3ad8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.415859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1260/2489] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:48.420007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.430100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e054fc-93e5-47fe-aa30-6061470698c3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.434966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.435617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.435894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.436167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] PATH_INFO: `/v3/projects/8064b32e949b4e0786dfafa41f5d03ed/users/502dbf1572604a0fb34b10ea2aa9191b/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.452343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8064b32e949b4e0786dfafa41f5d03ed, user_id=502dbf1572604a0fb34b10ea2aa9191b, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.455095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.471671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1230/2490] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:48 2022] PUT /identity/v3/projects/8064b32e949b4e0786dfafa41f5d03ed/users/502dbf1572604a0fb34b10ea2aa9191b/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:48.482059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.482272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.482561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.510906 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] MFA Rules not processed for user `502dbf1572604a0fb34b10ea2aa9191b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:48.512109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:48.581504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1261/2491] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:48.621005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.622874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:48.650858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:48.694962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08286e-5a56-4b5d-9ad0-48262eafd915 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.698738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.699464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.699749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.700025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.710968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.713087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419936-e96c-4b88-a5c8-940c5258823f service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.723715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1231/2492] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:48.741148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:48.811761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33b0d-1f0a-45d5-901e-9d51e0dc8146 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:48.817153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:48.817904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.818241 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.818522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] PATH_INFO: `/v3/users/502dbf1572604a0fb34b10ea2aa9191b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.819722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502dbf1572604a0fb34b10ea2aa9191b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:48.821711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:48.874104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502dbf1572604a0fb34b10ea2aa9191b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:48.881841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502dbf1572604a0fb34b10ea2aa9191b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:48.882459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502dbf1572604a0fb34b10ea2aa9191b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:48.883111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502dbf1572604a0fb34b10ea2aa9191b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:48.895909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502dbf1572604a0fb34b10ea2aa9191b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:48.904471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:48.910754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1262/2493] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:48 2022] DELETE /identity/v3/users/502dbf1572604a0fb34b10ea2aa9191b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:48.916106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:48.916390 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:48.916658 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:48.937446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:48.938753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:48.999031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1232/2494] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:49.032213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.033781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.092085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275ec44-577b-49c0-90d8-a60de55143d6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.095815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.096422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.096715 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.096968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.107636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.109718 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.121388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.122628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1263/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:49.122990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.215001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.216969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.219985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057e48e-3e40-4ecf-9ecb-7c2f1f09f004 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.226624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.228129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.228564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.228996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.231208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.237881 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73041d1b-073a-44d9-be23-d657fd633cc0 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.246389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.248030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.258810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1233/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:49.260245 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.262194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.262194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.262194 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] PATH_INFO: `/v3/users/69ab1bc171834c87bd0614050d92a7e9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.263045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ab1bc171834c87bd0614050d92a7e9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.265264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.328550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ab1bc171834c87bd0614050d92a7e9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.335364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ab1bc171834c87bd0614050d92a7e9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.335983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ab1bc171834c87bd0614050d92a7e9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.336593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ab1bc171834c87bd0614050d92a7e9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.348452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ab1bc171834c87bd0614050d92a7e9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.362173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1264/2497] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:49 2022] DELETE /identity/v3/users/69ab1bc171834c87bd0614050d92a7e9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:49.547640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.549105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.563879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75100780-2862-4a1e-962a-b3848b5346b1 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.629581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.631283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.675481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:49.683690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.685351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.685635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.685963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] PATH_INFO: `/v3/projects/8064b32e949b4e0786dfafa41f5d03ed` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.687247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8064b32e949b4e0786dfafa41f5d03ed)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.689427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.708553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125ee9e-fe70-4e79-9bdb-9cc5312e2dff admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.713689 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.714314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.714601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.714890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] PATH_INFO: `/v3/projects/a68067e070dd4da380f6378faba35193` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.715997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68067e070dd4da380f6378faba35193)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.717890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.734863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:49.740747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8064b32e949b4e0786dfafa41f5d03ed', 'request_id': 'req-578b15a2-d3b6-44dd-93ef-6afe098b87cd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.747995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8064b32e949b4e0786dfafa41f5d03ed', 'request_id': 'req-578b15a2-d3b6-44dd-93ef-6afe098b87cd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.755965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8064b32e949b4e0786dfafa41f5d03ed', 'request_id': 'req-578b15a2-d3b6-44dd-93ef-6afe098b87cd'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.766324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68067e070dd4da380f6378faba35193', 'request_id': 'req-1551a7c3-2571-40f3-998d-1e0ef48554c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.766791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] The token cache is being invalidate because project 8064b32e949b4e0786dfafa41f5d03ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:49.767083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.767707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-578b15a2-d3b6-44dd-93ef-6afe098b87cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.769212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1234/2498] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:49 2022] DELETE /identity/v3/projects/8064b32e949b4e0786dfafa41f5d03ed => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:49.773883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19840b7-bc83-4f25-b161-90c1d827c62b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.774169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19840b7-bc83-4f25-b161-90c1d827c62b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.774976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19840b7-bc83-4f25-b161-90c1d827c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.777070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68067e070dd4da380f6378faba35193', 'request_id': 'req-1551a7c3-2571-40f3-998d-1e0ef48554c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.786071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68067e070dd4da380f6378faba35193', 'request_id': 'req-1551a7c3-2571-40f3-998d-1e0ef48554c9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.797968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a19840b7-bc83-4f25-b161-90c1d827c62b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:49.799297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a19840b7-bc83-4f25-b161-90c1d827c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.802369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] The token cache is being invalidate because project a68067e070dd4da380f6378faba35193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:49.802810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.803566 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.805440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1265/2499] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:49 2022] DELETE /identity/v3/projects/a68067e070dd4da380f6378faba35193 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:49.809381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.811180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.867358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1235/2500] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:49.875314 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.875911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.876309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.886174 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a7c3-2571-40f3-998d-1e0ef48554c9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.891606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.892233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.892510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.892800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] PATH_INFO: `/v3/users/3367ae9ff0af477fb672ae4caa5fae77` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.893915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3367ae9ff0af477fb672ae4caa5fae77)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.895852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.896907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:49.898103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:49.907883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1236/2501] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:49.911873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.922351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914dc93-70fb-48e3-ada8-4fad72c45b37 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.927670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.928305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.928627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.928971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.929784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:49.930179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.932370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.945500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1237/2502] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:59:49.950281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:49.951018 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3367ae9ff0af477fb672ae4caa5fae77'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.959341 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3367ae9ff0af477fb672ae4caa5fae77'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.960405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3367ae9ff0af477fb672ae4caa5fae77'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.960803 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b996a-fc59-452e-9e05-e7331f0d6d19 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:49.961146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3367ae9ff0af477fb672ae4caa5fae77'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.966263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:49.966921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:49.967999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:49.967999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:49.968229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:49.970103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:49.975585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3367ae9ff0af477fb672ae4caa5fae77'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:49.994165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1266/2503] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:49 2022] DELETE /identity/v3/users/3367ae9ff0af477fb672ae4caa5fae77 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:50.002635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1238/2504] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:49 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:50.003835 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.005299 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.035354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:50.077234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d70b8c-6674-4106-bca1-f4569b007e2e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.137779 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.138475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.138848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.139116 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.139743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.141813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.165122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.166433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1267/2505] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:50.166877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.170526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.182991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205d08-f79c-4dd8-829d-2d2cf85b019a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.185749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6f89c-0095-41ef-8d04-c5cebe7eafe2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.192158 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.192808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.193117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.193390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.194038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.195675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.200208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.201085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.201286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.201579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] PATH_INFO: `/v3/projects/9372029bff8349249685682d58a87a15` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.203271 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9372029bff8349249685682d58a87a15)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.205552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.209044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1268/2506] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:50.213705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.224149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c26aa-f09e-443d-9e78-e4d0f5574f2f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.229046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.229712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.230056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.230393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.231172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.232818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.252957 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:50.260236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1269/2507] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:50.261500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9372029bff8349249685682d58a87a15', 'request_id': 'req-0fa119ed-dbff-43af-b59b-5f059932c17c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:50.264398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.269832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9372029bff8349249685682d58a87a15', 'request_id': 'req-0fa119ed-dbff-43af-b59b-5f059932c17c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:50.274926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5878cc-4927-43a9-9e4f-5aeccca9f946 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.281310 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9372029bff8349249685682d58a87a15', 'request_id': 'req-0fa119ed-dbff-43af-b59b-5f059932c17c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:50.298786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] The token cache is being invalidate because project 9372029bff8349249685682d58a87a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:50.298786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:50.298786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0fa119ed-dbff-43af-b59b-5f059932c17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:50.306485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1239/2508] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:50 2022] DELETE /identity/v3/projects/9372029bff8349249685682d58a87a15 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:50.307959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e8f3-4ca4-4d4f-af4e-e2c161efaff9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.308194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e8f3-4ca4-4d4f-af4e-e2c161efaff9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.308924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54e8f3-4ca4-4d4f-af4e-e2c161efaff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.331300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1e54e8f3-4ca4-4d4f-af4e-e2c161efaff9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:50.333417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1e54e8f3-4ca4-4d4f-af4e-e2c161efaff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.339432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.340105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.340384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.340651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] PATH_INFO: `/v3/projects/8ebf9fba55214a4f9f3973820d19dacc/users/ff275bf4958b414abc35115faef9666d/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.343959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1240/2509] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:50.351445 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.351731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.352118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.356006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebf9fba55214a4f9f3973820d19dacc, user_id=ff275bf4958b414abc35115faef9666d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.358658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b66018-d067-40e6-83fd-0179ee8bbbae admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.373298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:50.374654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.388250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1270/2510] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:50 2022] PUT /identity/v3/projects/8ebf9fba55214a4f9f3973820d19dacc/users/ff275bf4958b414abc35115faef9666d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:50.389799 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.390403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.390781 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.422017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] MFA Rules not processed for user `ff275bf4958b414abc35115faef9666d`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:50.423276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.431407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:50.441599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1241/2511] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:50.445754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.456041 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8fa3-2fc6-4cd4-b94a-7988acd2127d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.460742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381898-abba-4759-83ec-a9804187576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.461371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381898-abba-4759-83ec-a9804187576d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.461649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381898-abba-4759-83ec-a9804187576d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.462444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58381898-abba-4759-83ec-a9804187576d admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.463124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381898-abba-4759-83ec-a9804187576d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:50.463426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381898-abba-4759-83ec-a9804187576d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.465024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58381898-abba-4759-83ec-a9804187576d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.481259 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1242/2512] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:59:50.482229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381898-abba-4759-83ec-a9804187576d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.492091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58381898-abba-4759-83ec-a9804187576d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.492540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1271/2513] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:50.497087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.498188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.498188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.498387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.499025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.501058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.504646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.506035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.529132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1243/2514] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:50.533435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.564572 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9418-f9bc-4487-8815-f82a7c8aa72d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.594060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86255f-fb37-474c-8a01-0a9a3871b386 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.599688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.600290 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.600569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.600844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.601493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.604056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.614250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.614990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.615685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.615685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.626410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1244/2515] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:50.630373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.640522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d1636-33c4-4c01-bdf2-74260af1d9f6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.646016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.646621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.646901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.647197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.647835 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.650024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.659958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1245/2516] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:50.663685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.673184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b1045-be40-49ab-a3c6-bb42ff5d552b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.676856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.677841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.678297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.678432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.678717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.678995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.679618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.681209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32b331-691e-4f18-8eea-11ffaee67215 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.682473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:50.687842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1272/2517] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:50.692592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1246/2518] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:50.698615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.710855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c3528-a597-4be9-b42c-b0d6c52a3349 service glance] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.714646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.715305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.715584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.715857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] PATH_INFO: `/v3/projects/063edc4580c24dceb178fcd122955c75/users/c41e4a9d724a471a8ecb38c385fa3d6d/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.731043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] RBAC: Authorizing `identity:create_grant(project_id=063edc4580c24dceb178fcd122955c75, user_id=c41e4a9d724a471a8ecb38c385fa3d6d, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.733684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.736824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:50.748770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1273/2519] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:50 2022] PUT /identity/v3/projects/063edc4580c24dceb178fcd122955c75/users/c41e4a9d724a471a8ecb38c385fa3d6d/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:50.760438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.760724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.760996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.789106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] MFA Rules not processed for user `c41e4a9d724a471a8ecb38c385fa3d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:50.790346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:50.855901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1247/2520] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:50.860358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:50.941482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde5a9b-7c40-4715-9c26-cbc98c9edfce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:50.950370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:50.951024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:50.951305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:50.951580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:50.952252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:50.954199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:50.990875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1274/2521] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:50.995489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.010149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd55d04-56fb-4248-8199-4f026e04af2e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.082713 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.083912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.084206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.084709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.085530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.087754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.138270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1248/2522] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:50 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:51.141932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.152485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699fc51-a431-4570-a5ce-b4deadf6b456 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.157467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.158242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.158597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.158965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.159678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.161405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.175219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1275/2523] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:51.179232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.191506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a5a-7b5f-4892-8a56-4d03121daf3c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.197540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.198266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.198627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.198906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] PATH_INFO: `/v3/projects/455a12d0df754531a0f6e7a45dd66520/users/ec0f6fe3ae174567beef75631246566b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.215634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=455a12d0df754531a0f6e7a45dd66520, user_id=ec0f6fe3ae174567beef75631246566b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.220518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.237998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1249/2524] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:51 2022] PUT /identity/v3/projects/455a12d0df754531a0f6e7a45dd66520/users/ec0f6fe3ae174567beef75631246566b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:51.241976 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.254073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb429d1-2986-41da-8b6e-8fee68133b02 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.316205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.316818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.317104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.317372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.318013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.319741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bae91-6070-4110-ad4a-0835b3b8fc14 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.336382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1276/2525] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 06:59:51.337573 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.351227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d0764-98ad-4e84-bc75-4f88910dc4be admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.356922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.357757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.359260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.359643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] PATH_INFO: `/v3/projects/455a12d0df754531a0f6e7a45dd66520/users/ec0f6fe3ae174567beef75631246566b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.362241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455a12d0df754531a0f6e7a45dd66520, user_id=ec0f6fe3ae174567beef75631246566b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.365437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.379653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1250/2526] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:51 2022] PUT /identity/v3/projects/455a12d0df754531a0f6e7a45dd66520/users/ec0f6fe3ae174567beef75631246566b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:51.390361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.390888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.390999 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.423403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] MFA Rules not processed for user `ec0f6fe3ae174567beef75631246566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:51.424605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:51.479977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:51.504792 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1277/2527] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:51.516347 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:51.518144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:51.561932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:51.578943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccad8bb-cd56-47a1-a67a-e8b8a50eed36 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:51.582715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:51.583260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:51.583536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:51.583811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:51.592702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:51.594165 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7a806-d399-47c8-93f0-3e38c5d3ec85 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:51.605128 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1251/2528] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:52.037785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:52.100932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9ae87-712e-4f6f-9ca5-991e0902147d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:52.105961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:52.106639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:52.106927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:52.107197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] PATH_INFO: `/v3/users/c41e4a9d724a471a8ecb38c385fa3d6d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:52.108387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41e4a9d724a471a8ecb38c385fa3d6d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:52.110365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:52.162467 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41e4a9d724a471a8ecb38c385fa3d6d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.170808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41e4a9d724a471a8ecb38c385fa3d6d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.171525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41e4a9d724a471a8ecb38c385fa3d6d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.172170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41e4a9d724a471a8ecb38c385fa3d6d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.186078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41e4a9d724a471a8ecb38c385fa3d6d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.199868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1278/2529] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:52 2022] DELETE /identity/v3/users/c41e4a9d724a471a8ecb38c385fa3d6d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:52.204195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:52.204712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:52.205096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:52.231814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:52.233095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:52.297980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1252/2530] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:52.308301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:52.309837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:52.379246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b847322-21e3-4a59-ada1-e32baa5b23c6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:52.383297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:52.383984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:52.384257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:52.384519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:52.394078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:52.395663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:52.409005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1279/2531] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:52.685844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:52.687598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:52.701406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dac8a2-1e03-4a2c-ac20-2c13ce9c818c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:52.706382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:52.707075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:52.707351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:52.707621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] PATH_INFO: `/v3/projects/063edc4580c24dceb178fcd122955c75` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:52.708807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] RBAC: Authorizing `identity:delete_project(project_id=063edc4580c24dceb178fcd122955c75)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:52.710745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:52.759243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063edc4580c24dceb178fcd122955c75', 'request_id': 'req-59594118-a9ed-4228-a020-c8eadb28e71c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.765425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063edc4580c24dceb178fcd122955c75', 'request_id': 'req-59594118-a9ed-4228-a020-c8eadb28e71c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.772653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063edc4580c24dceb178fcd122955c75', 'request_id': 'req-59594118-a9ed-4228-a020-c8eadb28e71c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.782550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] The token cache is being invalidate because project 063edc4580c24dceb178fcd122955c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:52.782860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.783470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.784942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1253/2532] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:52 2022] DELETE /identity/v3/projects/063edc4580c24dceb178fcd122955c75 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:52.788868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:52.794019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:52.799815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9a5a4-2249-4f89-97cf-1b849580cf05 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:52.862391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:52.863007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:52.863283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:52.863551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] PATH_INFO: `/v3/users/ec0f6fe3ae174567beef75631246566b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:52.864670 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0f6fe3ae174567beef75631246566b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:52.866627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:52.914496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0f6fe3ae174567beef75631246566b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.922423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0f6fe3ae174567beef75631246566b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.923111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0f6fe3ae174567beef75631246566b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.923745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0f6fe3ae174567beef75631246566b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.940157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0f6fe3ae174567beef75631246566b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:52.952349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1280/2533] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:52 2022] DELETE /identity/v3/users/ec0f6fe3ae174567beef75631246566b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:53.225702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.227899 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.241351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59594118-a9ed-4228-a020-c8eadb28e71c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.302529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.304095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.308944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.309585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.309868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.310143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] PATH_INFO: `/v3/projects/455a12d0df754531a0f6e7a45dd66520` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.311384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455a12d0df754531a0f6e7a45dd66520)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.313298 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.326273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:53.366300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455a12d0df754531a0f6e7a45dd66520', 'request_id': 'req-633189da-2b40-4cc4-a038-052545773c16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:53.367768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53331b06-3843-40c8-9a66-e37fc7c997fb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.372361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455a12d0df754531a0f6e7a45dd66520', 'request_id': 'req-633189da-2b40-4cc4-a038-052545773c16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:53.395966 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455a12d0df754531a0f6e7a45dd66520', 'request_id': 'req-633189da-2b40-4cc4-a038-052545773c16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:53.409120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] The token cache is being invalidate because project 455a12d0df754531a0f6e7a45dd66520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:53.409370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:53.409999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-633189da-2b40-4cc4-a038-052545773c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:53.410705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:53.412139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1254/2534] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:53 2022] DELETE /identity/v3/projects/455a12d0df754531a0f6e7a45dd66520 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:53.418591 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6268d-8989-46b1-8497-689332d55648 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.418894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6268d-8989-46b1-8497-689332d55648 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.419274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6268d-8989-46b1-8497-689332d55648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.442087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-65b6268d-8989-46b1-8497-689332d55648 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:53.443353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-65b6268d-8989-46b1-8497-689332d55648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.447495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.448117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.448527 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.448802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.450815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.520600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.523927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.535661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1255/2535] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:53.539580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1281/2536] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:53.543934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.544214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.544484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.564750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:53.565771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.576167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1256/2537] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:53.582784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.597461 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4fb9d-7a84-45bf-9572-9489f7ad74fe service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.602077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.603282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.603552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.603816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.604452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:53.604746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.606328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.620225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1282/2538] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 06:59:53.624641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.635178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb1a93-99a7-4bcb-a4bd-fd12d30396a0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.639952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.640553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.640851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.641092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.641723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.643683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.682068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1257/2539] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:53 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:53.685924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.697458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d84c0-76e3-4169-90c7-a879c44512e5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.781869 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.783815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.785646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.786492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.786890 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.787630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.788485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.790740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.820389 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1283/2540] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:53 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:53.824725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.837314 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc69ae-c26a-4e40-9503-07f1b7b466d9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.842821 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.843598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.844073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.844643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.845448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.847395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.851971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654eb692-e757-43b9-93fb-4594c5022149 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.856285 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.857011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.857356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.857907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.860299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:53.861073 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1284/2541] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:53.866054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.878101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.879699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c796b6-8776-4b29-be54-e9c1f4e3ec69 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.880149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.885938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.886547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.887009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.887403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.888173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.889851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1258/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:53.889986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca46c49a-ba9c-408f-9f62-14eef2527963 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.900618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1285/2543] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:53.905582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:53.919862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc4598-7f32-44ce-88c8-33c5280cd4df service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:53.925289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:53.926047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.926413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.926781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] PATH_INFO: `/v3/projects/41cfbccbc8ad48d79bd2609c113edc17/users/98a07b295c3842068902f6bb8a58c0c2/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:53.930617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:53.943669 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cfbccbc8ad48d79bd2609c113edc17, user_id=98a07b295c3842068902f6bb8a58c0c2, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:53.947424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:53.967671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1259/2544] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:53 2022] PUT /identity/v3/projects/41cfbccbc8ad48d79bd2609c113edc17/users/98a07b295c3842068902f6bb8a58c0c2/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:53.979480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:53.979981 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:53.980325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.014640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] MFA Rules not processed for user `98a07b295c3842068902f6bb8a58c0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:54.015977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.046035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:54.084473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1286/2545] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:54.088913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.158514 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9666f1-77aa-48db-b0dd-d454c1a3ddc3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.165260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.166060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.166450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.166903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.167673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.169780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.201051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1260/2546] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:54 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:54.204936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.219875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9b3a-ea5c-423b-b01b-d4d2626e93bc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.272520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.275119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.296055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.296669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.296942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.297376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.298551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.301032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.324189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1287/2547] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:54 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:54.327746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.355087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b53fca-e0ab-44d3-9cbb-8a8ff2696d5b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.360357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.360978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.361258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.361533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.362182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.364988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.368848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536e765-3e94-41aa-b1eb-9514972f8e14 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.389129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1288/2548] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:54.389250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.389666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.390087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.390397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] PATH_INFO: `/v3/users/e702f6ad9e38492497e0ae33abb9aae7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.391813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e702f6ad9e38492497e0ae33abb9aae7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.391953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.393807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.403477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb3bf1-2a38-4b7d-b38f-d68023412255 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.409160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.409925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.410204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.410605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] PATH_INFO: `/v3/projects/ec097c5170e04b61892c4157afa0e3c3/users/ab23e8503b5c407ea87b8938b1498d16/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.427702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec097c5170e04b61892c4157afa0e3c3, user_id=ab23e8503b5c407ea87b8938b1498d16, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.431001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.453372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1289/2549] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:54 2022] PUT /identity/v3/projects/ec097c5170e04b61892c4157afa0e3c3/users/ab23e8503b5c407ea87b8938b1498d16/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:54.457142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.468536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e702f6ad9e38492497e0ae33abb9aae7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.482135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e702f6ad9e38492497e0ae33abb9aae7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.489047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e702f6ad9e38492497e0ae33abb9aae7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.490039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e702f6ad9e38492497e0ae33abb9aae7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.504793 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e702f6ad9e38492497e0ae33abb9aae7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.515868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:54.529517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1261/2550] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:54 2022] DELETE /identity/v3/users/e702f6ad9e38492497e0ae33abb9aae7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 06:59:54.540223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a97c-a827-423b-9a42-9d39197a74fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.573652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:54.617506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.619189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.619189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.619503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.620111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.621734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de3ef1-ac04-4613-8704-9a829f6bebc9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.632650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1290/2551] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:54.637203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.639288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.654894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b343a-6364-4eae-9532-85c46dedab01 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.660068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.660942 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.661276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.661554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] PATH_INFO: `/v3/projects/ec097c5170e04b61892c4157afa0e3c3/users/ab23e8503b5c407ea87b8938b1498d16/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.663589 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec097c5170e04b61892c4157afa0e3c3, user_id=ab23e8503b5c407ea87b8938b1498d16, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.666767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.678185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1262/2552] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:54 2022] PUT /identity/v3/projects/ec097c5170e04b61892c4157afa0e3c3/users/ab23e8503b5c407ea87b8938b1498d16/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:54.689043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.689323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.689594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.719686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] MFA Rules not processed for user `ab23e8503b5c407ea87b8938b1498d16`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:54.721274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.799995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1291/2553] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:54.801496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.803744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.812697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.819677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.882036 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc5427-bd5d-4ff3-9a26-9d3507e9834b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.885810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.887115 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.887474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.887747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.889402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3c0f9-4018-4b46-9208-b2831146c4d3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.894362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:54.895040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.895322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.895763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] PATH_INFO: `/v3/projects/1d431bdc9a2244dcb1bb589dee792d02` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:54.897027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.897206 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d431bdc9a2244dcb1bb589dee792d02)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:54.898460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.900275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:54.909310 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1292/2554] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:54.952809 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d431bdc9a2244dcb1bb589dee792d02', 'request_id': 'req-ed533d2d-c7a7-4371-8c19-52818be04b54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.957054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:54.958466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:54.959939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d431bdc9a2244dcb1bb589dee792d02', 'request_id': 'req-ed533d2d-c7a7-4371-8c19-52818be04b54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.968230 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d431bdc9a2244dcb1bb589dee792d02', 'request_id': 'req-ed533d2d-c7a7-4371-8c19-52818be04b54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.970505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2dd6-428b-4e7d-b547-f985e6a37df3 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:54.981079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] The token cache is being invalidate because project 1d431bdc9a2244dcb1bb589dee792d02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 06:59:54.981364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.982014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ed533d2d-c7a7-4371-8c19-52818be04b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:54.983555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1263/2555] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 06:59:54 2022] DELETE /identity/v3/projects/1d431bdc9a2244dcb1bb589dee792d02 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:54.989818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4d9f9-4380-4fda-9146-e0de1fff24eb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:54.990107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4d9f9-4380-4fda-9146-e0de1fff24eb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:54.990363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4d9f9-4380-4fda-9146-e0de1fff24eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.011110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e7c4d9f9-4380-4fda-9146-e0de1fff24eb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:55.012411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e7c4d9f9-4380-4fda-9146-e0de1fff24eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:55.031733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.032833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.033220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.033782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.035649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:55.047529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:55.073116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1264/2556] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:55.080907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.081189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.081460 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.095564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:55.100029 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.101755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.103815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:55.105280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:55.112380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1293/2557] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 06:59:55.117746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1265/2558] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 06:59:55.118902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.129691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185e02-0cd2-40ed-a9fe-198938930fe8 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.134886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.135473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.135777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.136041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.136672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 06:59:55.136966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.138540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.150324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1294/2559] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 06:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 06:59:55.155397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.166978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71061c7-a4ac-4ed2-9b94-33df071f432b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.171833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.172457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.172801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.173160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.173891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.175919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.210614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1266/2560] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 06:59:55 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 06:59:55.212428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.222403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca6ca-5b76-4681-b743-0dc74a45082d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.284001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.284673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.284949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.285213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.285840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.287785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.311515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1295/2561] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 06:59:55 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 06:59:55.315590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.326017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bed97-87f0-414c-8625-54ad57137076 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.330863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.331447 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.331723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.331988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.332626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.334307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.344456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1267/2562] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:55.348498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.360336 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a82b6f-e183-4a40-9d9e-9ccedbfea40d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.365325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.365974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.366259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.366531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.367203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.369524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a572fd0-1fbd-4573-bc25-92abc121f2a4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.394252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1296/2563] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 06:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 06:59:55.396212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.413946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3249f-0cab-47e4-b8b3-c0c6304b5e0b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.420526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.421370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.421715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.423510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] PATH_INFO: `/v3/projects/f707372e1ec948dcb7e89b9a046cccfc/users/b997560479c642e0ac43c0b08a1bf157/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.459420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f707372e1ec948dcb7e89b9a046cccfc, user_id=b997560479c642e0ac43c0b08a1bf157, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.463580 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.477880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1268/2564] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 06:59:55 2022] PUT /identity/v3/projects/f707372e1ec948dcb7e89b9a046cccfc/users/b997560479c642e0ac43c0b08a1bf157/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:55.483080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.483619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.483619 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.503805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:55.505020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:55.565675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1297/2565] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:55.576843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:55.578714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:55.646281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde3531-b69a-4c5d-b7e3-34403ade3262 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:55.651596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:55.652209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:55.652491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:55.652766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:55.662633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:55.664076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec1858-2900-44aa-ae7f-8c25c9306583 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:55.674788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1269/2566] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 06:59:59.106263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:59.108049 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:59.126836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d432862-f27f-484a-8fc1-a1ca9794b4a6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:59.140892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:59.145918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:59.146568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:59.147043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] PATH_INFO: `/v3/users/98a07b295c3842068902f6bb8a58c0c2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:59.148644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a07b295c3842068902f6bb8a58c0c2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:59.152922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:59.189602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 06:59:59.248227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a07b295c3842068902f6bb8a58c0c2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:59.262338 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a07b295c3842068902f6bb8a58c0c2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:59.263953 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a07b295c3842068902f6bb8a58c0c2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:59.264569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a07b295c3842068902f6bb8a58c0c2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:59.289216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a07b295c3842068902f6bb8a58c0c2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 06:59:59.305291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1298/2567] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 06:59:59 2022] DELETE /identity/v3/users/98a07b295c3842068902f6bb8a58c0c2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 06:59:59.310384 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:59.310911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:59.311807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:59.339147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 06:59:59.340404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:59.413192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1270/2568] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 06:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 06:59:59.424181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 06:59:59.428262 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 06:59:59.511580 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6563-f274-4140-9139-06575f5d80e6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 06:59:59.515983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 06:59:59.516863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 06:59:59.517497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 06:59:59.517713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 06:59:59.538061 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 06:59:59.540485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a77bfb-d160-465e-be4b-54c4358de4e5 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 06:59:59.552217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1299/2569] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 06:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:00.125888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.128032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.142186 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2aeb0-7757-483b-adc2-b4d1bea69d69 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.151158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.152135 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.152594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.153208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] PATH_INFO: `/v3/projects/41cfbccbc8ad48d79bd2609c113edc17` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.154927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cfbccbc8ad48d79bd2609c113edc17)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.157082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.185962 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.186517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.186517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.216572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:00.224959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cfbccbc8ad48d79bd2609c113edc17', 'request_id': 'req-d98fba1a-b744-4007-b76f-4d670cbd9225'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.225957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] MFA Rules not processed for user `b997560479c642e0ac43c0b08a1bf157`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:00.227817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.234000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cfbccbc8ad48d79bd2609c113edc17', 'request_id': 'req-d98fba1a-b744-4007-b76f-4d670cbd9225'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.249527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cfbccbc8ad48d79bd2609c113edc17', 'request_id': 'req-d98fba1a-b744-4007-b76f-4d670cbd9225'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.271077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] The token cache is being invalidate because project 41cfbccbc8ad48d79bd2609c113edc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:00.271370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.272027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.273860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1271/2570] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:00 2022] DELETE /identity/v3/projects/41cfbccbc8ad48d79bd2609c113edc17 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:00.277524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.278934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.327856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1300/2571] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:00.332204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.333654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.356196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fba1a-b744-4007-b76f-4d670cbd9225 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.361093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.361838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.362176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.362537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] PATH_INFO: `/v3/users/ab23e8503b5c407ea87b8938b1498d16` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.363093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:00.363946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab23e8503b5c407ea87b8938b1498d16)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.365975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.413874 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46fe76-4080-4e1f-82f8-2fb3201b37ae None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.419886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.420759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.421039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.421360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.422019 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.423072 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab23e8503b5c407ea87b8938b1498d16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.424375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.432696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab23e8503b5c407ea87b8938b1498d16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.433388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab23e8503b5c407ea87b8938b1498d16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.434213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab23e8503b5c407ea87b8938b1498d16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.443241 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab23e8503b5c407ea87b8938b1498d16'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:00.457677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1301/2572] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:00:00 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:00.461676 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.463401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.479800 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1272/2573] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:00 2022] DELETE /identity/v3/users/ab23e8503b5c407ea87b8938b1498d16 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:00:00.539231 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c81db-36e8-4ccd-99a0-c5f6544a4efe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.625558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.633579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.633747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.634034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.634809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.636800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.666626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1302/2574] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:00:00 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:00:00.670981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.683365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f203-ee21-4f0f-a71d-697f6e07f689 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.689050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.694067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.694299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.694767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.695331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.697715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.710212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1273/2575] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:00.715069 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.725427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e84da-dde8-4cd3-b1a7-91184b2fd3dc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.730320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.731030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.731305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.731574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] PATH_INFO: `/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5/users/320d32dac8e04f438d5be8f72e3c460a/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.752304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1728fc9adc45c3aa39654a0bb257e5, user_id=320d32dac8e04f438d5be8f72e3c460a, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.754988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.770933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1303/2576] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:00 2022] PUT /identity/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5/users/320d32dac8e04f438d5be8f72e3c460a/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:00:00.774929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.785052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa5ddd-6d09-4bd4-a74c-b0fb6a51b866 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.827275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.829045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:00.838756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:00.864595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:00.865088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.865768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.865928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.866202 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.866872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.868538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.886218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1274/2577] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:00.887791 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:00.900584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870359-2386-4536-8d5d-d0b99127ab8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.905673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.906608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.907197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.907653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] PATH_INFO: `/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5/users/320d32dac8e04f438d5be8f72e3c460a/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.911029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1728fc9adc45c3aa39654a0bb257e5, user_id=320d32dac8e04f438d5be8f72e3c460a, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.913960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722160f-9c68-433d-b8e8-d97858b56ee0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:00.915449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c94c05-f63d-418c-bd6f-dcb516d85ac7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.921663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:00.922444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:00.922755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:00.923079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] PATH_INFO: `/v3/projects/ec097c5170e04b61892c4157afa0e3c3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:00.924547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec097c5170e04b61892c4157afa0e3c3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:00.926894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:00.928866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1275/2578] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:00 2022] PUT /identity/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5/users/320d32dac8e04f438d5be8f72e3c460a/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:00.995612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec097c5170e04b61892c4157afa0e3c3', 'request_id': 'req-79739497-8617-45a7-85b9-3280cf1d1ea0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:01.003436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec097c5170e04b61892c4157afa0e3c3', 'request_id': 'req-79739497-8617-45a7-85b9-3280cf1d1ea0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:01.014051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec097c5170e04b61892c4157afa0e3c3', 'request_id': 'req-79739497-8617-45a7-85b9-3280cf1d1ea0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:01.029128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] The token cache is being invalidate because project ec097c5170e04b61892c4157afa0e3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:01.029778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:01.030815 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79739497-8617-45a7-85b9-3280cf1d1ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:01.032416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1304/2579] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:00 2022] DELETE /identity/v3/projects/ec097c5170e04b61892c4157afa0e3c3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:01.037413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.037691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.038035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.060458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:01.061732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:01.158814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1276/2580] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:01.162035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.162324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.162628 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.200512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:01.201825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:01.213276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1305/2581] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:01.217849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.232970 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08b474-2add-42a0-8d81-4a2923b87939 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.246242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.246906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.247176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.247558 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.248088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:00:01.248389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.249981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.267375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1277/2582] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:00:01.271754 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.282745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d8df-b66a-4180-9898-558e11748f43 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.290783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.291450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.291712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.292000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.292643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.294719 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.360355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1306/2583] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:01.365090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.383496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f0b6f-6411-4db7-ad89-2a5d3a8422d4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.457837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.458462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.459214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.459685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.460478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.463777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.490100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1278/2584] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:01.494652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.507535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da062a1-514b-428c-8875-028bdf5f03e7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.516129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.516922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.517298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.517663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.518450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.520333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.534499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1307/2585] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:01.538850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.551683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6e1c6-14ea-4538-aab1-4510ca7f11be admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.560214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.561777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.562039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.562338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.563614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.565409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8126ce4-6d3d-4f6a-a765-799e5d259c3e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.582636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1279/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:01.586940 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.599345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1198e-470c-40f7-8a57-03c3e4833d24 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.604747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.605438 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.605716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.605985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] PATH_INFO: `/v3/projects/5f9c8182dd6c4c25866a85dda6efce80/users/130096a3d73e43c7ad6bc171a68731a4/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.622244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9c8182dd6c4c25866a85dda6efce80, user_id=130096a3d73e43c7ad6bc171a68731a4, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.625117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.643862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1308/2587] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:01 2022] PUT /identity/v3/projects/5f9c8182dd6c4c25866a85dda6efce80/users/130096a3d73e43c7ad6bc171a68731a4/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:00:01.656383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e8985-53b2-4ab8-a049-458204161368 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.656681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e8985-53b2-4ab8-a049-458204161368 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.656956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e8985-53b2-4ab8-a049-458204161368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.691272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-071e8985-53b2-4ab8-a049-458204161368 None None] MFA Rules not processed for user `130096a3d73e43c7ad6bc171a68731a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:01.692339 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-071e8985-53b2-4ab8-a049-458204161368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:01.707723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:01.785404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1280/2588] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:01.790801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.857164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:01.864862 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe4fd0-f772-40c8-bf60-3ef9f937e581 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:01.873099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:01.874426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:01.874847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:01.875433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:01.876763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:01.879371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:01.924348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e8985-53b2-4ab8-a049-458204161368 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:01.931501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1309/2589] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:01.938138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e8985-53b2-4ab8-a049-458204161368 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.025014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.025724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.026074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.026415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.027148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.029153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.053794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1281/2590] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:00:01 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:02.057858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:02.069153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13431090-1778-4744-8516-836708a3eb52 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.074814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.075841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.076063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.076501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.077365 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.079090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.095052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1310/2591] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:02.096252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:02.107821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96a24e-75ea-4da1-8a28-fe73930ee333 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.112590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.113256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.113532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.113936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] PATH_INFO: `/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2/users/fd2f137cc1464c8fac63cdb645d7c31b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.131827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91925af7c5e401d931b9a09ba1c3ec2, user_id=fd2f137cc1464c8fac63cdb645d7c31b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.136164 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.162914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1282/2592] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:02 2022] PUT /identity/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2/users/fd2f137cc1464c8fac63cdb645d7c31b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:00:02.164741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:02.176103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca4bdd-5dba-4bf0-a7f1-68fa19850a49 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.250830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.252259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.252595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.253094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.254291 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.257593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b97ba-5f98-4c80-8aac-9ebeb4dcf249 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.270008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1311/2593] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:00:02.274174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:02.286101 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f5621-e1e6-4ff5-8bd2-b71275f0f596 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.294953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.295824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.296172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.297090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] PATH_INFO: `/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2/users/fd2f137cc1464c8fac63cdb645d7c31b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.299431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e91925af7c5e401d931b9a09ba1c3ec2, user_id=fd2f137cc1464c8fac63cdb645d7c31b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.302257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.330073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1283/2594] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:02 2022] PUT /identity/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2/users/fd2f137cc1464c8fac63cdb645d7c31b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:02.330323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.330323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.330323 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.372768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] MFA Rules not processed for user `fd2f137cc1464c8fac63cdb645d7c31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:02.374167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:02.477434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1312/2595] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:00:02.488125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:02.490103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:02.558201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1473a8-60dd-4337-8356-10372a9f9cf2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:02.562371 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:02.562765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:02.563143 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:02.563516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:02.575811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:02.577448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02956d9b-ccbe-4dbf-9386-33b9eecd5a7f service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:02.589464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1284/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:03.135499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:03.186611 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:03.210514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8457f79-21e7-4bd4-ae80-e163fa2c9ec8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:03.215825 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:03.216627 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:03.217293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:03.217660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] PATH_INFO: `/v3/users/130096a3d73e43c7ad6bc171a68731a4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:03.220391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=130096a3d73e43c7ad6bc171a68731a4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:03.228303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:03.323349 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '130096a3d73e43c7ad6bc171a68731a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:03.335114 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '130096a3d73e43c7ad6bc171a68731a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:03.335808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '130096a3d73e43c7ad6bc171a68731a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:03.338140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '130096a3d73e43c7ad6bc171a68731a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:03.364961 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '130096a3d73e43c7ad6bc171a68731a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:03.406321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1313/2597] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:03 2022] DELETE /identity/v3/users/130096a3d73e43c7ad6bc171a68731a4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:03.412565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:03.412565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:03.412565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:03.432788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:03.434121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:03.440901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:03.581775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1285/2598] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:03.592027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:03.593459 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:03.682448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae8dfd-2b69-4ded-b614-fd3dfa1af853 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:03.687415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:03.688109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:03.688564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:03.688945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:03.702123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:03.704782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:03.718122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1314/2599] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:04.194958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:04.197005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:04.217973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d201f-f896-4a5b-a6da-468890c78483 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:04.228470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:04.230085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:04.230671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:04.231215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] PATH_INFO: `/v3/projects/5f9c8182dd6c4c25866a85dda6efce80` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:04.234227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9c8182dd6c4c25866a85dda6efce80)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:04.237965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:04.352185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9c8182dd6c4c25866a85dda6efce80', 'request_id': 'req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.368812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9c8182dd6c4c25866a85dda6efce80', 'request_id': 'req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.393441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9c8182dd6c4c25866a85dda6efce80', 'request_id': 'req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.415530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] The token cache is being invalidate because project 5f9c8182dd6c4c25866a85dda6efce80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:04.415530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.416753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.418877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1286/2600] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:04 2022] DELETE /identity/v3/projects/5f9c8182dd6c4c25866a85dda6efce80 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:04.422768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:04.436496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e24962-f039-4fbf-a9fd-0e411dabf38f service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:04.596148 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:04.597039 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:04.597412 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:04.597777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] PATH_INFO: `/v3/users/fd2f137cc1464c8fac63cdb645d7c31b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:04.599057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2f137cc1464c8fac63cdb645d7c31b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:04.601221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:04.608488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:04.703250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2f137cc1464c8fac63cdb645d7c31b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.721851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2f137cc1464c8fac63cdb645d7c31b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.722960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2f137cc1464c8fac63cdb645d7c31b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.724947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2f137cc1464c8fac63cdb645d7c31b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.755343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-eba4f231-657c-4ff3-8d28-f537eb991b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2f137cc1464c8fac63cdb645d7c31b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:04.792277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1315/2601] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:04 2022] DELETE /identity/v3/users/fd2f137cc1464c8fac63cdb645d7c31b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:05.305868 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:05.307944 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:05.348951 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b08fe0-c5fc-4f5f-97a4-2bee38d020ba admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:05.548555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:05.550408 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:05.550752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:05.551031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] PATH_INFO: `/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:05.554747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e91925af7c5e401d931b9a09ba1c3ec2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:05.556697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:05.703074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:05.733664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e91925af7c5e401d931b9a09ba1c3ec2', 'request_id': 'req-e566ef20-15d5-4104-917f-c08c903f8e08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:05.760619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e91925af7c5e401d931b9a09ba1c3ec2', 'request_id': 'req-e566ef20-15d5-4104-917f-c08c903f8e08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:05.783555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e91925af7c5e401d931b9a09ba1c3ec2', 'request_id': 'req-e566ef20-15d5-4104-917f-c08c903f8e08'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:05.819059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] The token cache is being invalidate because project e91925af7c5e401d931b9a09ba1c3ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:05.819526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:05.822508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e566ef20-15d5-4104-917f-c08c903f8e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:05.824617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1287/2602] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:05 2022] DELETE /identity/v3/projects/e91925af7c5e401d931b9a09ba1c3ec2 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:05.831159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:05.836584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:05.836960 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:05.881354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:05.885818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:06.031995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1316/2603] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:06.041119 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.041391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.041662 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.072251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:06.073474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:06.090243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1288/2604] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:06.094624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.111573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0961dea-8192-4174-8c19-8f20ca9b792a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.127002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.127705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.127878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.128235 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.129362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:00:06.129775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.131610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.174630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1317/2605] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:00:06.179345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.191872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d4955-6b17-4147-bf01-79998e045a63 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.197814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.199306 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.200235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.200576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.201563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.203710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.294670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1289/2606] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:00:06 2022] POST /identity/v3/projects => generated 395 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:06.299020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.316750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0ddf0-fb14-4301-b233-63b9c3710c4e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.454396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.457587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.458701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.458701 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.459610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.462942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.520673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:06.534747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1318/2607] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:00:06 2022] POST /identity/v3/users => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:00:06.535555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.560446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adf8c7-d831-48be-97a8-ccd65c1129c6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.575676 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.577025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.577552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.578013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.578959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.586715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.607997 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.632293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1290/2608] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:06.635308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67dc3d-3a5b-4e82-b301-9309048a4cd2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.649153 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.649787 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.650066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.650339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.651095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.652882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c833889-34fa-4df4-bcd6-08d882a41389 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.665751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1319/2609] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:00:06.669610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.680988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e064b-07d6-45fb-9a66-455ffca865da admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:06.686975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:06.687582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.687856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.688127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] PATH_INFO: `/v3/projects/0f34253dfc4c43759edadfb671de0e6e/users/9dbe3d04b96c41eab562ac1635053b12/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.705499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f34253dfc4c43759edadfb671de0e6e, user_id=9dbe3d04b96c41eab562ac1635053b12, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:06.708231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:06.727817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1291/2610] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:00:06 2022] PUT /identity/v3/projects/0f34253dfc4c43759edadfb671de0e6e/users/9dbe3d04b96c41eab562ac1635053b12/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:06.738623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79134-01c3-4698-90b6-e18041e008c0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:06.739082 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79134-01c3-4698-90b6-e18041e008c0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:06.739426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e79134-01c3-4698-90b6-e18041e008c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:06.788648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-14e79134-01c3-4698-90b6-e18041e008c0 None None] MFA Rules not processed for user `9dbe3d04b96c41eab562ac1635053b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:06.791110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-14e79134-01c3-4698-90b6-e18041e008c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:06.921125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1320/2611] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:06.933632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:06.935456 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:07.052246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43c562-3d01-420e-8000-c656628f367d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:07.059842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:07.064258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:07.064258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:07.064258 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:07.070324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b420-9de4-4f40-9cce-cfbcb82801f0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:07.072010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b420-9de4-4f40-9cce-cfbcb82801f0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:07.072297 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b420-9de4-4f40-9cce-cfbcb82801f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:07.078738 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:07.080204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:07.100478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1292/2612] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:06 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:07.111878 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b184b420-9de4-4f40-9cce-cfbcb82801f0 None None] MFA Rules not processed for user `320d32dac8e04f438d5be8f72e3c460a`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:07.113252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b184b420-9de4-4f40-9cce-cfbcb82801f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:07.257557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1321/2613] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:00:07.274994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:07.277653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:07.380113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:07.390994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e91ff6-9883-457c-80d9-19727276ff04 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:07.398434 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:07.399699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:07.400068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:07.400400 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:07.402978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:07.480068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:07.481777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:07.491040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1293/2614] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:12.099434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bbf94-7022-4d65-97c2-1dcc1ba71cce None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:12.099942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bbf94-7022-4d65-97c2-1dcc1ba71cce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:12.100053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bbf94-7022-4d65-97c2-1dcc1ba71cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:12.160570 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-221bbf94-7022-4d65-97c2-1dcc1ba71cce None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:12.161796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-221bbf94-7022-4d65-97c2-1dcc1ba71cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:12.187775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:12.236526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1322/2615] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 07:00:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:12.284397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:12.285812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:12.398438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e064f38-0eee-491a-9894-5b6d8279dcab service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:12.404333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:12.404936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:12.405213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:12.405515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:12.413887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:12.415501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dd85f-f424-42e5-b689-81a5404d5c60 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:12.423383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1294/2616] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:00:15.822243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:15.822851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:15.822851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:15.824223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1323/2617] 10.222.0.20 () {54 vars in 912 bytes} [Sun Sep 18 07:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 18 07:00:15.831413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfec93c-1597-4f6e-b639-920509b0a59a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:15.831998 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfec93c-1597-4f6e-b639-920509b0a59a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:15.832234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfec93c-1597-4f6e-b639-920509b0a59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:15.886146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cdfec93c-1597-4f6e-b639-920509b0a59a None None] MFA Rules not processed for user `cc74075ebcdd4916aacec44e87e86064`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:15.887692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cdfec93c-1597-4f6e-b639-920509b0a59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:15.951149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:15.966399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1295/2618] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 07:00:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:15.991827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:15.993604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:16.014236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d158f-84e8-4055-98cd-b6e9fa70c426 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:16.020495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:16.021404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:16.021686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:16.022263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:16.035067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:16.037005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:16.050675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1324/2619] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:17.365578 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:17.366440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:17.366891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:17.393403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:17.394884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:17.403442 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1296/2620] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 07:00:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:00:17.418699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:17.435726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49a79e-3181-425d-82b1-d5d7b9ded460 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:17.441421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:17.442259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:17.442840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:17.443175 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:17.453010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:17.455048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:17.464290 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1325/2621] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:00:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:00:17.722913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:17.821831 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023370d1-5b02-457b-81e3-d74d84d69970 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:17.826732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:17.827428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:17.827770 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:17.828107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] PATH_INFO: `/v3/users/9dbe3d04b96c41eab562ac1635053b12` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:17.829471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbe3d04b96c41eab562ac1635053b12)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:17.831688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:17.931563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbe3d04b96c41eab562ac1635053b12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:17.942643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbe3d04b96c41eab562ac1635053b12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:17.943585 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbe3d04b96c41eab562ac1635053b12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:17.944843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbe3d04b96c41eab562ac1635053b12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:17.953893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fb74077a-a55e-4c5d-babc-d49800f17c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbe3d04b96c41eab562ac1635053b12'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:17.975004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:17.977057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:17.984544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1297/2622] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:17 2022] DELETE /identity/v3/users/9dbe3d04b96c41eab562ac1635053b12 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:17.989120 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:17.989404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:17.989887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:18.013681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:18.014968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:18.066075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ce1d-837a-436c-a004-a556a646daac service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:18.090035 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1298/2623] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:00:18.100280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:18.102723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:18.114197 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:18.131607 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:18.132221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:18.132496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:18.132763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:18.134624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:18.154533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:18.156207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:18.176321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1326/2624] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:18.195709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7f10-a529-49d3-8016-c0052c562e21 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:18.199879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:18.200520 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:18.200795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:18.201064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:18.212632 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:18.214780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846964b0-17cd-40d8-9cc3-bae30a421430 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:18.231740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1299/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:00:18.562740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:18.565443 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:18.593869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9a82b-0a75-47da-bcf8-cfdea08afa40 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:18.605697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:18.606597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:18.607076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:18.607612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] PATH_INFO: `/v3/projects/0f34253dfc4c43759edadfb671de0e6e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:18.609209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f34253dfc4c43759edadfb671de0e6e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:18.612469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:18.679149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f34253dfc4c43759edadfb671de0e6e', 'request_id': 'req-9c62c025-9b94-4e27-8ae9-5e05a006146e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:18.686064 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f34253dfc4c43759edadfb671de0e6e', 'request_id': 'req-9c62c025-9b94-4e27-8ae9-5e05a006146e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:18.687939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:18.695214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f34253dfc4c43759edadfb671de0e6e', 'request_id': 'req-9c62c025-9b94-4e27-8ae9-5e05a006146e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:18.708548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] The token cache is being invalidate because project 0f34253dfc4c43759edadfb671de0e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:18.708853 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:18.709568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:18.712424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1327/2626] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:18 2022] DELETE /identity/v3/projects/0f34253dfc4c43759edadfb671de0e6e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:19.196697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1473-a864-458e-8847-872e7273b6f1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:19.197292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1473-a864-458e-8847-872e7273b6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:19.197292 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1473-a864-458e-8847-872e7273b6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:19.228594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a7ac1473-a864-458e-8847-872e7273b6f1 None None] MFA Rules not processed for user `00399347c76d451d94d45f4edc3cf416`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:19.229836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a7ac1473-a864-458e-8847-872e7273b6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:19.287061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1300/2627] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:19.659982 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:19.661496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:19.731646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62c025-9b94-4e27-8ae9-5e05a006146e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:19.737256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:19.738157 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:19.738361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:19.738665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] PATH_INFO: `/v3/users/ff275bf4958b414abc35115faef9666d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:19.739804 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff275bf4958b414abc35115faef9666d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:19.741729 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:19.796382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff275bf4958b414abc35115faef9666d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:19.810933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff275bf4958b414abc35115faef9666d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:19.812259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff275bf4958b414abc35115faef9666d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:19.814176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff275bf4958b414abc35115faef9666d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:19.829836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff275bf4958b414abc35115faef9666d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:19.844124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1328/2628] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:19 2022] DELETE /identity/v3/users/ff275bf4958b414abc35115faef9666d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:19.848491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:19.848846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:19.849189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:19.877263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:19.877263 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:19.948544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1301/2629] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:19.959360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:19.961700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:20.030139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8f52b-592c-4fa0-8ec1-85f494a92a87 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:20.035305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:20.035932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:20.036212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:20.036476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:20.057062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:20.058703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:20.069969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1329/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:20.403136 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:20.405616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:20.442236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef48a1-08f9-4a5c-a35d-d50e2d44630c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:20.450066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:20.451884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:20.452653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:20.453257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] PATH_INFO: `/v3/projects/8ebf9fba55214a4f9f3973820d19dacc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:20.454864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebf9fba55214a4f9f3973820d19dacc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:20.459725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:20.530859 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:20.536117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebf9fba55214a4f9f3973820d19dacc', 'request_id': 'req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:20.547893 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebf9fba55214a4f9f3973820d19dacc', 'request_id': 'req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:20.557001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebf9fba55214a4f9f3973820d19dacc', 'request_id': 'req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:20.569360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] The token cache is being invalidate because project 8ebf9fba55214a4f9f3973820d19dacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:20.569783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:20.571036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e2d88f1-954c-408c-bd8b-0cd2b86c6f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:20.573364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1302/2631] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:20 2022] DELETE /identity/v3/projects/8ebf9fba55214a4f9f3973820d19dacc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:27.165011 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:27.166655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:27.177376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:27.181126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717c7e-85c2-4ecc-aeac-53f301621267 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:27.245015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:27.245646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:27.245926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:27.246197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] PATH_INFO: `/v3/users/fd862cc69267470580ef23e0223742e0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:27.247355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd862cc69267470580ef23e0223742e0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:27.249263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:27.297159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd862cc69267470580ef23e0223742e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.304802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd862cc69267470580ef23e0223742e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.305591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd862cc69267470580ef23e0223742e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.306189 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd862cc69267470580ef23e0223742e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.317921 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd862cc69267470580ef23e0223742e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.330340 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1330/2632] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:27 2022] DELETE /identity/v3/users/fd862cc69267470580ef23e0223742e0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:27.456933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:27.457496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:27.457496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:27.476939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] MFA Rules not processed for user `b997560479c642e0ac43c0b08a1bf157`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:00:27.478232 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:27.533381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:27.534988 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:27.559751 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1303/2633] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:00:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:00:27.629164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13aa6-516c-4a91-a5d6-2817e5874935 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:27.634354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:27.635226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:27.635530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:27.635770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] PATH_INFO: `/v3/projects/2af8940da97b4298a92e1d45f70cfe29` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:27.636915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af8940da97b4298a92e1d45f70cfe29)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:27.639080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:27.692101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:27.721440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af8940da97b4298a92e1d45f70cfe29', 'request_id': 'req-79801670-4071-46c8-ba42-c9c1ee44f33f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.728337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af8940da97b4298a92e1d45f70cfe29', 'request_id': 'req-79801670-4071-46c8-ba42-c9c1ee44f33f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.737190 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af8940da97b4298a92e1d45f70cfe29', 'request_id': 'req-79801670-4071-46c8-ba42-c9c1ee44f33f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.748164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] The token cache is being invalidate because project 2af8940da97b4298a92e1d45f70cfe29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:27.748457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.749089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.750900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1331/2634] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:27 2022] DELETE /identity/v3/projects/2af8940da97b4298a92e1d45f70cfe29 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:00:27.754904 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:27.765958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144f5c0-4ced-4591-bfe8-2139fe8f203a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:27.822570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:27.823218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:27.823493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:27.823762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] PATH_INFO: `/v3/users/94b8657a6b6042dca12872ed16854596` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:27.837867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b8657a6b6042dca12872ed16854596)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:27.839741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:27.886567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b8657a6b6042dca12872ed16854596'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.895307 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b8657a6b6042dca12872ed16854596'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.896147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b8657a6b6042dca12872ed16854596'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.896908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b8657a6b6042dca12872ed16854596'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.905127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b8657a6b6042dca12872ed16854596'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:27.909598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:27.925739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1304/2635] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:00:27 2022] DELETE /identity/v3/users/94b8657a6b6042dca12872ed16854596 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:28.163355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:28.165165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:28.178638 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79801670-4071-46c8-ba42-c9c1ee44f33f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:28.267319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:28.267957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:28.268374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:28.268649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] PATH_INFO: `/v3/projects/203da2a4b1de47bdae454841113a2fa9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:28.280610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] RBAC: Authorizing `identity:delete_project(project_id=203da2a4b1de47bdae454841113a2fa9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:28.282593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:28.348375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203da2a4b1de47bdae454841113a2fa9', 'request_id': 'req-a970d6be-458a-4507-8b7a-1d061534791c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:28.355354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203da2a4b1de47bdae454841113a2fa9', 'request_id': 'req-a970d6be-458a-4507-8b7a-1d061534791c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:28.363510 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203da2a4b1de47bdae454841113a2fa9', 'request_id': 'req-a970d6be-458a-4507-8b7a-1d061534791c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:28.374330 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] The token cache is being invalidate because project 203da2a4b1de47bdae454841113a2fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:00:28.374650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:28.375542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:00:28.377209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1332/2636] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:00:28 2022] DELETE /identity/v3/projects/203da2a4b1de47bdae454841113a2fa9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:00:35.294177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:35.295638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:35.383325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5491a1-6328-4cf2-a57e-da948e416c39 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:35.452235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:35.452852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:35.453218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:35.453576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:35.455512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:35.470328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:35.472670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:35.486376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1305/2637] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:38.371394 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=13,keystone:SELECT=69,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:43.306818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:43.308304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:43.324658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970d6be-458a-4507-8b7a-1d061534791c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:43.330214 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:43.330851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:43.331130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:43.331404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:43.333138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:43.346145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:43.347860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff7913-7d3c-47c4-b7cb-f619f6f6de82 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:43.370778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1333/2638] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:45.281636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:00:45.283075 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:45.339173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a6677-43e4-4f6b-b494-d596ae885ac3 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:00:45.344712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:00:45.345674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:00:45.345954 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:00:45.346221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:00:45.347956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:00:45.358020 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:45.542502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:00:45.544909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:00:45.556107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1306/2639] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:00:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:00:53.356139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:00:55.550981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=24 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:31.247233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:31.247762 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:31.247895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:31.267559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:01:31.268777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:31.324476 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1334/2640] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 07:01:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:01:31.372651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:31.374406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:31.448201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b45572-e5af-4470-a622-08a02a577053 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:31.454511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:31.455622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:31.456471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:31.456643 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:31.470749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:31.472363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:31.481341 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1307/2641] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:01:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:01:41.317185 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=25 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:41.478012 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=32 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:44.091583 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:44.092985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.104817 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60972ff-c592-4a1d-8254-2f7158d5cc86 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:44.108396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:44.108996 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:44.109272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:44.109540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:44.111213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.120851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:44.122264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:44.130243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1335/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:01:44.255705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:44.257185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.315565 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1a668-1b76-4834-ac58-cd5ddf9a8e81 service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:44.319054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e5439bf8f1f480faef63316021d4b27', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:44.319912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:44.320190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:44.320840 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:44.322141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.336536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:44.337987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:44.348758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1308/2643] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:01:44.418068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:44.426525 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c296d-b5fb-4d88-9a3d-1cbf5cfa6345 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:44.430226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:44.433550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:44.433889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:44.434173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:44.436055 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.446056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:44.447530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:44.456070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1336/2644] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:01:44.543374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:44.553028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab1e77-6b0b-4839-96bd-2a1996ff1462 service glance] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:44.556777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:44.557452 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:44.557785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:44.558114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:44.559941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.573471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:44.574850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ba364-d08a-4911-a0b8-5a2e3381eb6b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:44.586784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1309/2645] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:01:44.647163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:44.648552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.716012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e699c3-d6bc-4fc0-af30-5ac8d59e9a99 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:44.720968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:44.721567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:44.721840 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:44.722106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:44.723839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:44.734946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:44.736512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:44.746238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1337/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:01:46.303134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7509-edcc-4547-9cd8-f8c42faec2e5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:46.303651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7509-edcc-4547-9cd8-f8c42faec2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:46.303756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d7509-edcc-4547-9cd8-f8c42faec2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:46.322178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e03d7509-edcc-4547-9cd8-f8c42faec2e5 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:01:46.323383 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e03d7509-edcc-4547-9cd8-f8c42faec2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:01:46.330561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1310/2647] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 07:01:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:01:46.342423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:01:46.353895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d95e7-3d33-408c-b66f-0523fb053c2c service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:01:46.358756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:01:46.359368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:01:46.359643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:01:46.359911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:01:46.368326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:01:46.369901 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:01:46.378129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1338/2648] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:01:56.316941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=35 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:01:56.374822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=50 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:00.968336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:00.968875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:00.968875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:00.988087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:00.989308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:00.996918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1311/2649] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 07:02:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:01.009447 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:01.020001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aa4ce-8aa8-411b-93d8-03a9af2cadc4 service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:01.025324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4350-56da-4555-b611-153af8012177 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:01.025936 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4350-56da-4555-b611-153af8012177 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:01.026219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4350-56da-4555-b611-153af8012177 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:01.026492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4350-56da-4555-b611-153af8012177 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:01.034919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4350-56da-4555-b611-153af8012177 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:01.036629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4350-56da-4555-b611-153af8012177 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:01.045626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1339/2650] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 18 07:02:10.982784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:11.042201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:34.145746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:34.154877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e63d-1193-4b57-9840-7e0dca94ab74 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:34.158325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:34.158930 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:34.159205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:34.159471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:34.161140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:34.174086 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:34.175539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:34.186061 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1312/2651] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:35.368424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4350-56da-4555-b611-153af8012177 service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:35.378014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4350-56da-4555-b611-153af8012177 service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:35.381745 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:35.382334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:35.382631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:35.382902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:35.384665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:35.403128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:35.404567 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:35.416107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1340/2652] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:41.990079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:41.998865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb199e-d983-48fc-85db-7a0ce12be01c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:42.002355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:42.002971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:42.003250 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:42.003517 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:42.005981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:42.090302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:42.091760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:42.110626 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1313/2653] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:45.039516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.040943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:45.054909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78e4d3-a301-4aee-8cd4-84e698e40e1d service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.059911 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.060501 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.060777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.061046 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.062956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:45.074439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.076093 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.095286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1341/2654] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:02:45.111106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.113597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:45.128986 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7540acc-e753-40c0-aea2-6e5e3ad42682 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.138977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.139627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.140153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.140454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] PATH_INFO: `/v3/users/00399347c76d451d94d45f4edc3cf416` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.141574 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] RBAC: Authorizing `identity:delete_user(user_id=00399347c76d451d94d45f4edc3cf416)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.143647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.192106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00399347c76d451d94d45f4edc3cf416'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.199802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00399347c76d451d94d45f4edc3cf416'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.200357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00399347c76d451d94d45f4edc3cf416'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.200952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00399347c76d451d94d45f4edc3cf416'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.209356 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cfaafe49-092f-42a6-ab0b-5af52b4da4da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00399347c76d451d94d45f4edc3cf416'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.227995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1314/2655] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:02:45 2022] DELETE /identity/v3/users/00399347c76d451d94d45f4edc3cf416 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:02:45.414480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.430924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0808e6f-a6f9-4bf4-b097-544f48cc6132 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.493587 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.494223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.494500 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.494899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] PATH_INFO: `/v3/projects/6fbf7eeb158e4a72bd8133faa1366e0d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.496048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fbf7eeb158e4a72bd8133faa1366e0d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.498221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.544995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fbf7eeb158e4a72bd8133faa1366e0d', 'request_id': 'req-600a944a-93a2-47da-a23c-a5e1135e7565'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.553357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fbf7eeb158e4a72bd8133faa1366e0d', 'request_id': 'req-600a944a-93a2-47da-a23c-a5e1135e7565'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.561325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fbf7eeb158e4a72bd8133faa1366e0d', 'request_id': 'req-600a944a-93a2-47da-a23c-a5e1135e7565'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.572281 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] The token cache is being invalidate because project 6fbf7eeb158e4a72bd8133faa1366e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:02:45.572479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.573066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-600a944a-93a2-47da-a23c-a5e1135e7565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:45.578640 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1342/2656] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:02:45 2022] DELETE /identity/v3/projects/6fbf7eeb158e4a72bd8133faa1366e0d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:45.580273 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.580559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.580848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.627508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:45.656937 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:45.657683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:45.729331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1315/2657] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:02:45.737099 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.737382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.737650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.758382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:45.765803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:45.777067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1343/2658] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:45.781361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.792039 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a35f6d-99b3-4c07-bb8c-3eb44c240e89 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.797369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.798084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.798362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.798673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.799322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:02:45.799623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.801288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.813431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1316/2659] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 07:02:45.817790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.827385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143c1ec-1bee-4d21-b1ad-20acfd399cdb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.832823 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.833428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.833699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.833964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.834615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.836797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.875655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1344/2660] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:02:45 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:45.879852 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.889846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2af99f-20b5-4132-9661-466816e4c3d7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:45.948199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:45.948813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:45.949091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:45.949381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:45.949991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:45.951934 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:45.982392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1317/2661] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:02:45 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:45.983642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:45.997750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ee9c1-126c-4e70-a667-80ae3b318146 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:46.006795 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:46.007691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:46.007966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:46.008236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:46.009083 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:46.011309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:46.026822 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1345/2662] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:46.027863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:46.041146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099086f-79c5-4732-bbee-15ebe9dc9cb8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:46.047827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:46.048444 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:46.048723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:46.049043 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:46.049796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:46.051552 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c1ec4-d948-4af0-a4bb-01536bff7614 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:46.072538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1318/2663] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:46.073622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:46.090707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63db1-860d-4bd8-9118-fb2639f5e5cf admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:46.098767 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:46.099788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:46.100170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:46.100537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] PATH_INFO: `/v3/projects/2b163e3d06f44f438429d4c4d7938efc/users/519d638b49f54728892c31be780d062e/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:46.121402 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:46.132044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b163e3d06f44f438429d4c4d7938efc, user_id=519d638b49f54728892c31be780d062e, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:46.136601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:46.163021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1346/2664] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:46 2022] PUT /identity/v3/projects/2b163e3d06f44f438429d4c4d7938efc/users/519d638b49f54728892c31be780d062e/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:46.168221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24a818-b94e-4719-bd24-4f647b28333c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:46.169021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24a818-b94e-4719-bd24-4f647b28333c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:46.169453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24a818-b94e-4719-bd24-4f647b28333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:46.218170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2c24a818-b94e-4719-bd24-4f647b28333c None None] MFA Rules not processed for user `519d638b49f54728892c31be780d062e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:46.220251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c24a818-b94e-4719-bd24-4f647b28333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:46.292146 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:46.313112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1319/2665] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:46.326160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:46.327933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:46.429829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62408d-7243-4aa8-98e6-6b05dadc1925 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:46.435669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:46.436277 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:46.436553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:46.436830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:46.445756 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:46.447364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:46.466797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1347/2666] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:47.280562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1ae1-d7f9-4525-a4ef-9c60294a103f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:47.281010 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1ae1-d7f9-4525-a4ef-9c60294a103f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:47.281112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d1ae1-d7f9-4525-a4ef-9c60294a103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:47.299496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-168d1ae1-d7f9-4525-a4ef-9c60294a103f None None] MFA Rules not processed for user `b997560479c642e0ac43c0b08a1bf157`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:47.301174 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-168d1ae1-d7f9-4525-a4ef-9c60294a103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:47.354234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1320/2667] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:49.217839 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:49.298156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff2f0f-00bd-4543-af25-af63fed6e19c service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:49.303377 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:49.304005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:49.304276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:49.304559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] PATH_INFO: `/v3/users/519d638b49f54728892c31be780d062e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:49.305658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519d638b49f54728892c31be780d062e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:49.307622 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:49.356585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519d638b49f54728892c31be780d062e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:49.365492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519d638b49f54728892c31be780d062e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:49.366532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519d638b49f54728892c31be780d062e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:49.367169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519d638b49f54728892c31be780d062e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:49.380776 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519d638b49f54728892c31be780d062e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:49.386753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:49.398505 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1348/2668] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:02:49 2022] DELETE /identity/v3/users/519d638b49f54728892c31be780d062e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:49.402900 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:49.403533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:49.403533 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:49.424478 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:49.425991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:49.500927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1321/2669] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:02:49.511526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:49.513005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:49.611651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41374cca-1346-41b1-bd1e-4513eb51d930 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:49.617852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:49.619211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:49.619846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:49.620292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:49.633292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:49.635847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07461-ed04-403d-a4c0-373927989b68 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:49.648419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1349/2670] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:50.068180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.069726 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:50.083289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875905e-8001-47d5-9dea-d2dd57806dee None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.092987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.093652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.093932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.094205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] PATH_INFO: `/v3/projects/2b163e3d06f44f438429d4c4d7938efc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.095431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b163e3d06f44f438429d4c4d7938efc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.097395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.150780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b163e3d06f44f438429d4c4d7938efc', 'request_id': 'req-0cc6fa26-069c-4a21-8181-2624fd33d122'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:50.158366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b163e3d06f44f438429d4c4d7938efc', 'request_id': 'req-0cc6fa26-069c-4a21-8181-2624fd33d122'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:50.168441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b163e3d06f44f438429d4c4d7938efc', 'request_id': 'req-0cc6fa26-069c-4a21-8181-2624fd33d122'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:50.182005 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] The token cache is being invalidate because project 2b163e3d06f44f438429d4c4d7938efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:02:50.182515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:50.183336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0cc6fa26-069c-4a21-8181-2624fd33d122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:50.185819 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1322/2671] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:02:50 2022] DELETE /identity/v3/projects/2b163e3d06f44f438429d4c4d7938efc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:50.190444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.190867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.192337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.212913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:50.214353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:50.304199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1350/2672] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:02:50.310784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.311068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.311337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.342353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:50.344023 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:50.355015 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1323/2673] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:50.358970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.369796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fec2f5-0627-4395-a1c8-6a825df131b9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.374768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.375374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.376164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.376164 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.376551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:02:50.376846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.379451 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.395225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1351/2674] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:02:50.399708 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.410260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35c0b-4553-4938-8752-091e2549ee65 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.415600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.416396 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.416839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.417295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.418096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.420204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.431886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:50.459424 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1324/2675] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:50.463463 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.475348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b146e-c242-4e72-9135-afd08af2e8a2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.541252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.541937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.542217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.542483 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.543183 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.545223 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.573838 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1352/2676] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:02:50.577616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.589046 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f064073-ba06-4a1b-b485-0988a68b0e6a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.598623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.599300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.599553 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.599816 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.600582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.602173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.612949 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1325/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:02:50.616679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.630003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa1e1c-2e96-4a50-ab24-1117e85b5c94 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.635560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.636227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.636589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.636957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.637605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.639626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c1b5-9a21-4f78-a98c-581c35f21f04 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.651507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1353/2678] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:50.657685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.670331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939a142-3977-479a-b7ee-c58c3171ddcc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.696085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.696779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.697060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.697342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] PATH_INFO: `/v3/projects/8e2e098d93d04bbcb04d70ebf413c23e/users/1ff467a75a5e4642b0cf65ed1671cede/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.720160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2e098d93d04bbcb04d70ebf413c23e, user_id=1ff467a75a5e4642b0cf65ed1671cede, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.723527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.738927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1326/2679] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:50 2022] PUT /identity/v3/projects/8e2e098d93d04bbcb04d70ebf413c23e/users/1ff467a75a5e4642b0cf65ed1671cede/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:50.749899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.750427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.750844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.760704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:50.791665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] MFA Rules not processed for user `1ff467a75a5e4642b0cf65ed1671cede`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:50.793671 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:50.874092 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1354/2680] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:50.878686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:50.940268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f165bc-2f8b-4393-8da8-bb837f0627ad admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:50.946184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:50.946830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:50.948030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:50.948030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:50.948254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:50.950112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:50.986288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1327/2681] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:50.990735 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.005257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f0809-7a4a-4ad7-9f71-73e767bf0dc5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.064926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.065533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.065807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.066104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.066773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.068685 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.090206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1355/2682] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:02:50 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:02:51.094225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.105225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2d53b-1392-4909-9bca-879e569308a8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.109856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.110448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.110758 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.111034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.111675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.113264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.124787 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1328/2683] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:02:51.128922 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.139071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf0a2f-a2db-4001-970c-c5932d6d5190 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.143764 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.144928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.145209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.145482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] PATH_INFO: `/v3/projects/825ece1c99064810842c3aa8e3cec321/users/fc00fc4af44241c5a0d302ab2b526ab7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.162074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=825ece1c99064810842c3aa8e3cec321, user_id=fc00fc4af44241c5a0d302ab2b526ab7, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.164827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.182406 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1356/2684] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:51 2022] PUT /identity/v3/projects/825ece1c99064810842c3aa8e3cec321/users/fc00fc4af44241c5a0d302ab2b526ab7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:51.186266 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.196929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c5c6d-8864-4433-ae02-93528c9ff4e1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.255440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.256044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.256321 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.256607 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.257231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.258855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f99f1c9-baef-4bbf-bf11-df8721f2050c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.268750 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1329/2685] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:02:51.273591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.284136 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e40f8-4c4d-4210-bb93-438914b9f8a0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.288995 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.289648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.289927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.290201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] PATH_INFO: `/v3/projects/825ece1c99064810842c3aa8e3cec321/users/fc00fc4af44241c5a0d302ab2b526ab7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.292234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=825ece1c99064810842c3aa8e3cec321, user_id=fc00fc4af44241c5a0d302ab2b526ab7, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.294970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.305408 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1357/2686] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:51 2022] PUT /identity/v3/projects/825ece1c99064810842c3aa8e3cec321/users/fc00fc4af44241c5a0d302ab2b526ab7/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:51.373965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.374245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.374512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.403801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] MFA Rules not processed for user `fc00fc4af44241c5a0d302ab2b526ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:51.405026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:51.417019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:51.479369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1330/2687] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:51.492700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:51.494656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:51.552993 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:51.559900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e382346-f91e-44aa-bc3a-3a6a34affbe7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:51.564301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:51.564926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:51.565236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:51.565507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:51.576376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:51.577904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc5a1df-b57c-4b33-9726-4dbd172687c0 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:51.589450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1358/2688] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:52.205842 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:52.267265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f521d6-9452-4f84-9613-03fd2bdcb0d9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:52.271598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:52.272204 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:52.272477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:52.272743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] PATH_INFO: `/v3/users/1ff467a75a5e4642b0cf65ed1671cede` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:52.273839 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff467a75a5e4642b0cf65ed1671cede)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:52.275757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:52.322220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff467a75a5e4642b0cf65ed1671cede'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.329410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff467a75a5e4642b0cf65ed1671cede'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.330054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff467a75a5e4642b0cf65ed1671cede'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.330741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff467a75a5e4642b0cf65ed1671cede'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.337602 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff467a75a5e4642b0cf65ed1671cede'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.354754 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1331/2689] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:02:52 2022] DELETE /identity/v3/users/1ff467a75a5e4642b0cf65ed1671cede => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:52.359356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1963a-8928-4240-912e-303cbf159015 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:52.359832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1963a-8928-4240-912e-303cbf159015 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:52.359938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1963a-8928-4240-912e-303cbf159015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:52.379041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-05b1963a-8928-4240-912e-303cbf159015 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:52.380239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-05b1963a-8928-4240-912e-303cbf159015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:52.442215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1359/2690] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:52.457019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:52.458661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:52.491235 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:52.517528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ea35c-1491-41c9-bff1-de883ecfaae5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:52.521536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:52.522125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:52.522399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:52.522704 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:52.531410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:52.532826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:52.542697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1332/2691] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:52.806198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1963a-8928-4240-912e-303cbf159015 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:52.807856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-05b1963a-8928-4240-912e-303cbf159015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:52.822554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1963a-8928-4240-912e-303cbf159015 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:52.827392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:52.828361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:52.828361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:52.828625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] PATH_INFO: `/v3/projects/8e2e098d93d04bbcb04d70ebf413c23e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:52.829699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2e098d93d04bbcb04d70ebf413c23e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:52.832448 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:52.879733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2e098d93d04bbcb04d70ebf413c23e', 'request_id': 'req-aa54b945-6335-4ed6-ac33-2129ddc3a04d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.885644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2e098d93d04bbcb04d70ebf413c23e', 'request_id': 'req-aa54b945-6335-4ed6-ac33-2129ddc3a04d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.892598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2e098d93d04bbcb04d70ebf413c23e', 'request_id': 'req-aa54b945-6335-4ed6-ac33-2129ddc3a04d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.903357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] The token cache is being invalidate because project 8e2e098d93d04bbcb04d70ebf413c23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:02:52.903648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.904265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:52.905723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1360/2692] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:02:52 2022] DELETE /identity/v3/projects/8e2e098d93d04bbcb04d70ebf413c23e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:52.910343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:52.920114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd526fa-12a2-4291-b75e-a531c277046b service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:52.974958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:52.975577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:52.975854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:52.976123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] PATH_INFO: `/v3/users/fc00fc4af44241c5a0d302ab2b526ab7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:52.977231 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc00fc4af44241c5a0d302ab2b526ab7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:52.979171 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.024771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc00fc4af44241c5a0d302ab2b526ab7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.032916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc00fc4af44241c5a0d302ab2b526ab7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.033636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc00fc4af44241c5a0d302ab2b526ab7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.034368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc00fc4af44241c5a0d302ab2b526ab7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.042941 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e0358d35-7b96-4f9f-951b-88b72686dee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc00fc4af44241c5a0d302ab2b526ab7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.060990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1333/2693] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:02:52 2022] DELETE /identity/v3/users/fc00fc4af44241c5a0d302ab2b526ab7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:53.411800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.413526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:53.427632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54b945-6335-4ed6-ac33-2129ddc3a04d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.486283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.486945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.487226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.487497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] PATH_INFO: `/v3/projects/825ece1c99064810842c3aa8e3cec321` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.488652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] RBAC: Authorizing `identity:delete_project(project_id=825ece1c99064810842c3aa8e3cec321)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.490551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.506562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:53.538040 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825ece1c99064810842c3aa8e3cec321', 'request_id': 'req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.543984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825ece1c99064810842c3aa8e3cec321', 'request_id': 'req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.551362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825ece1c99064810842c3aa8e3cec321', 'request_id': 'req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.561429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] The token cache is being invalidate because project 825ece1c99064810842c3aa8e3cec321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:02:53.561712 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.562324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc1cc01-1f83-44b8-8e3a-73cabafc027a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:02:53.563845 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1361/2694] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:02:53 2022] DELETE /identity/v3/projects/825ece1c99064810842c3aa8e3cec321 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:53.569413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.569883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.570030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.590173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:53.591450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:53.636454 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:53.652714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1334/2695] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:53.660269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.660557 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.660832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.680254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:53.681454 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:53.689424 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1362/2696] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:53.693978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.703503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6839d3-adca-4bc2-a305-808ae24641d5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.708579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.709158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.709435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.709703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.710335 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:02:53.710667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.712269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.723624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1335/2697] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:02:53.727849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.736984 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f460613-2dbb-49c4-88d2-2d4d1ad91daf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.741588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.742172 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.742473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.742771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.743396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.745334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.775458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1363/2698] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:02:53 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:02:53.779642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.789563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dfb98-57ea-45be-8fd4-e637c69bbaf0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.846195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.846729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.847014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.847289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.847909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.849805 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.874144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1336/2699] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:02:53 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:53.879094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.889307 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5434969-6aab-44dc-87eb-735ac39467ce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.894220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.894842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.895122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.895390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.896030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.897608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.910155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1364/2700] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:53.911634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.922252 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaade4-a855-44ae-80df-f1ee4e1f6571 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.927328 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.927909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.928181 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.928446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.929076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.930694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8dbfb-51dd-4e24-b2b6-904e0ee5eb28 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.941907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1337/2701] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:53.945937 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:53.956038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b347-1fc4-435b-af8f-15ec33e8386a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:53.960739 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:53.961375 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:53.961654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:53.961926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] PATH_INFO: `/v3/projects/cee98f54503e4f219ee01458c4c7b807/users/457896382d3640b08fb2c6c02fd05a73/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:53.976125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee98f54503e4f219ee01458c4c7b807, user_id=457896382d3640b08fb2c6c02fd05a73, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:53.978819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:53.992470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1365/2702] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:53 2022] PUT /identity/v3/projects/cee98f54503e4f219ee01458c4c7b807/users/457896382d3640b08fb2c6c02fd05a73/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:54.004425 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.004717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.004984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.031742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] MFA Rules not processed for user `457896382d3640b08fb2c6c02fd05a73`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:54.032928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:54.100781 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1338/2703] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:54.105256 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.165043 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93de74-9e21-4bd2-9724-44bd118ea9ef admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.169832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.170431 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.170736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.171007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.171667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.173643 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.200578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1366/2704] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:02:54 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:54.204629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.215482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a647b-0491-4268-8f9b-9e9facd0db6d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.260994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:54.275616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.276217 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.276495 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.276765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.277407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.279362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.302336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1339/2705] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:02:54 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:02:54.306197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.316816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7b2ed-17e8-42c3-8e11-19f32d74c8e6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.321608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.322322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.322700 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.323035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.323740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.325790 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.337085 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1367/2706] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:54.341411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.351874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1352b0-9488-443c-8fa8-31d1ad4849cb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.356515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.357228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.357569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.357903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] PATH_INFO: `/v3/projects/23b9fe45f3bd4544898ab5b231b10563/users/1c309a495db74780a6f4a0e8fb0d765b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.373556 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b9fe45f3bd4544898ab5b231b10563, user_id=1c309a495db74780a6f4a0e8fb0d765b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.376503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.391477 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1340/2707] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:54 2022] PUT /identity/v3/projects/23b9fe45f3bd4544898ab5b231b10563/users/1c309a495db74780a6f4a0e8fb0d765b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:54.397268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.407351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f745d1b-bd62-4052-954a-cff73151d88e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.417355 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:02:54.465842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.466480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.466784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.467057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.467680 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.469263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc6490-8c43-418a-ae6b-fd25d0349fe1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.479433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1368/2708] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:02:54.483521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.494099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b8d0c-d86d-4115-b284-35666e11becf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.498865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.499494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.499766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.500031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] PATH_INFO: `/v3/projects/23b9fe45f3bd4544898ab5b231b10563/users/1c309a495db74780a6f4a0e8fb0d765b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.501918 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b9fe45f3bd4544898ab5b231b10563, user_id=1c309a495db74780a6f4a0e8fb0d765b, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.504577 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.514470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1341/2709] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:02:54 2022] PUT /identity/v3/projects/23b9fe45f3bd4544898ab5b231b10563/users/1c309a495db74780a6f4a0e8fb0d765b/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:02:54.524844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.525129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.525417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.553071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] MFA Rules not processed for user `1c309a495db74780a6f4a0e8fb0d765b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:02:54.554275 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:54.619462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1369/2710] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:02:54.630317 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:54.632418 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:02:54.698154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1055c14-20ae-4c55-bd7b-c97f058b48da admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:54.701784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:54.702374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:54.702688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:54.702963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:54.758642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:54.760079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:54.769466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1342/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:02:58.984584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:02:58.995044 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19917c00-3443-494e-b0f9-fb2f2e9f22dc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:02:58.998885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:02:58.999473 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:02:58.999746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:02:59.000012 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:02:59.008985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:02:59.010404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471aa7e-34f1-4b2e-beee-c4e7758208a0 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:02:59.020356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1370/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:02:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:03:03.905838 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:03.953279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:03.972707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7e6fd-bde9-4bd7-ab75-d4318f4eb414 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:03.977581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:03.978203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:03.978480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:03.978780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] PATH_INFO: `/v3/users/457896382d3640b08fb2c6c02fd05a73` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:03.979898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457896382d3640b08fb2c6c02fd05a73)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:03.981841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:04.031157 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457896382d3640b08fb2c6c02fd05a73'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.039660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457896382d3640b08fb2c6c02fd05a73'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.040350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457896382d3640b08fb2c6c02fd05a73'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.040977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457896382d3640b08fb2c6c02fd05a73'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.048485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457896382d3640b08fb2c6c02fd05a73'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.070052 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1343/2713] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:03 2022] DELETE /identity/v3/users/457896382d3640b08fb2c6c02fd05a73 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:04.074456 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:04.074932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:04.075052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:04.098197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:04.099604 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:04.159955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1371/2714] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:04.169720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:04.171172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:04.230971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae506b1-3c0a-485b-bb56-62e5125ebcd6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:04.234663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:04.235262 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:04.235541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:04.235811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:04.245107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:04.246541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:04.257218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1344/2715] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:04.543351 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:04.544847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:04.557699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49d2a-80ce-4f32-847f-7abe9e74e488 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:04.562679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:04.563331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:04.563608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:04.563875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] PATH_INFO: `/v3/projects/cee98f54503e4f219ee01458c4c7b807` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:04.565243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee98f54503e4f219ee01458c4c7b807)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:04.567879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:04.585796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:04.617541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee98f54503e4f219ee01458c4c7b807', 'request_id': 'req-3432ff70-798e-403d-a406-048e1397943c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.624293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee98f54503e4f219ee01458c4c7b807', 'request_id': 'req-3432ff70-798e-403d-a406-048e1397943c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.634705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee98f54503e4f219ee01458c4c7b807', 'request_id': 'req-3432ff70-798e-403d-a406-048e1397943c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.647228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] The token cache is being invalidate because project cee98f54503e4f219ee01458c4c7b807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:04.647535 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.648282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.650224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1372/2716] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:04 2022] DELETE /identity/v3/projects/cee98f54503e4f219ee01458c4c7b807 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:04.654270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:04.664365 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0d9f9-1622-4962-805b-1eab7a76389c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:04.722699 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:04.723854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:04.724130 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:04.724402 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] PATH_INFO: `/v3/users/1c309a495db74780a6f4a0e8fb0d765b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:04.726812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c309a495db74780a6f4a0e8fb0d765b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:04.727472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:04.733355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:04.780746 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c309a495db74780a6f4a0e8fb0d765b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.789469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c309a495db74780a6f4a0e8fb0d765b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.790099 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c309a495db74780a6f4a0e8fb0d765b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.790767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c309a495db74780a6f4a0e8fb0d765b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.798299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c309a495db74780a6f4a0e8fb0d765b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:04.817494 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1345/2717] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:04 2022] DELETE /identity/v3/users/1c309a495db74780a6f4a0e8fb0d765b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:03:04.895409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:04.896956 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.002392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432ff70-798e-403d-a406-048e1397943c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.073667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.074419 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.074942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.075522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.077742 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.095744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.097734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd8d24-2e80-4e4c-9246-c96a802041c1 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.114243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1373/2718] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:05.118698 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.120274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.199334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43796e-7953-45da-b2f0-91ab22d0cabb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.204624 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.206465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.206822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.207102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] PATH_INFO: `/v3/projects/23b9fe45f3bd4544898ab5b231b10563` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.209260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b9fe45f3bd4544898ab5b231b10563)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.211267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.259851 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b9fe45f3bd4544898ab5b231b10563', 'request_id': 'req-a1902468-7eff-49cf-bd20-dc1f56939410'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:05.266201 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b9fe45f3bd4544898ab5b231b10563', 'request_id': 'req-a1902468-7eff-49cf-bd20-dc1f56939410'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:05.275863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b9fe45f3bd4544898ab5b231b10563', 'request_id': 'req-a1902468-7eff-49cf-bd20-dc1f56939410'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:05.287848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] The token cache is being invalidate because project 23b9fe45f3bd4544898ab5b231b10563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:05.288127 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:05.288898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a1902468-7eff-49cf-bd20-dc1f56939410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:05.291026 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1346/2719] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:05 2022] DELETE /identity/v3/projects/23b9fe45f3bd4544898ab5b231b10563 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:05.297063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.297063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.297282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.316928 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:05.318134 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.343575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:05.380266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1374/2720] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:05.389069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.389212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.389505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.411598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:05.413057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.421212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1347/2721] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:05.425596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.435958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f4dd0-070d-4a1d-9d79-86355ddc3e8e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.440696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.441304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.442074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.442343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.443026 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:03:05.443328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.444933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.456989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1375/2722] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:03:05.462319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.473744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55bfbf-64db-46ea-b458-4a81d298bf05 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.478794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.479475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.479815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.480148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.480850 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.482854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.489272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:05.516100 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1348/2723] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:03:05.520916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.531123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0017-4c28-47e9-b03a-b08ef2b022b6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.611807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.612450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.612731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.613003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.614211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.616225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.638171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1376/2724] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:05 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:03:05.642394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.652948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b59f-2a7c-4214-a47e-bead1e44585a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.657534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.658126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.658397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.658689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.659324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.661581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.671492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1349/2725] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:05.675621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.686947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3585a3b-516f-466b-9bc3-b187b90621b1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.691598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.692197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.692485 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.692773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.693430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.695091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5ade7-403f-439a-a0cb-946dea09c272 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.705733 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1377/2726] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:05.709993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.719963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d177aa-f6f3-44e0-b633-33f3f3e57b04 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.725147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.725855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.726194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.726529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] PATH_INFO: `/v3/projects/5579c190401e40e4a47a2540402ac4a3/users/9d4fb5edba60498193b148466a919396/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.742084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5579c190401e40e4a47a2540402ac4a3, user_id=9d4fb5edba60498193b148466a919396, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.744901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.758518 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1350/2727] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:05 2022] PUT /identity/v3/projects/5579c190401e40e4a47a2540402ac4a3/users/9d4fb5edba60498193b148466a919396/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:05.771610 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.771943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.772242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.800130 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] MFA Rules not processed for user `9d4fb5edba60498193b148466a919396`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:05.802021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.866426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1378/2728] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:05.877635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:05.879345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:05.943462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf659-a90d-4aac-9ded-e2e23c8f498d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:05.947155 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:05.947757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:05.948034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:05.948302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:05.958464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:05.959936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9faaf6e-1809-4125-99a7-c10f75719670 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:05.972516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1351/2729] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:14.413803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:14.440276 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:14.477520 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6696e4-31cd-4b14-90e1-fb044039767e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:14.482104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:14.482773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:14.483051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:14.483322 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] PATH_INFO: `/v3/users/9d4fb5edba60498193b148466a919396` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:14.484439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4fb5edba60498193b148466a919396)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:14.486386 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:14.530763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4fb5edba60498193b148466a919396'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:14.538120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4fb5edba60498193b148466a919396'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:14.538796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4fb5edba60498193b148466a919396'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:14.539439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4fb5edba60498193b148466a919396'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:14.551294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4fb5edba60498193b148466a919396'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:14.564538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1379/2730] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:14 2022] DELETE /identity/v3/users/9d4fb5edba60498193b148466a919396 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:14.569030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:14.569469 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:14.569568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:14.587794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:14.588978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:14.647007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1352/2731] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:03:14.657119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:14.658538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:14.714533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47205-82eb-443b-b1af-94324df63e0b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:14.718220 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:14.718886 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:14.719180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:14.719452 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:14.728777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:14.730206 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749919ae-4221-439b-b271-5c123841b18a service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:14.740298 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1380/2732] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:15.003091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.004519 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:15.025635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7d478-deba-4f06-a22d-5ad6bc7b1f5f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.031002 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.031539 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.031821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.032085 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] PATH_INFO: `/v3/projects/5579c190401e40e4a47a2540402ac4a3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.033257 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5579c190401e40e4a47a2540402ac4a3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.035348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.082936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5579c190401e40e4a47a2540402ac4a3', 'request_id': 'req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:15.084623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:15.091921 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5579c190401e40e4a47a2540402ac4a3', 'request_id': 'req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:15.103277 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5579c190401e40e4a47a2540402ac4a3', 'request_id': 'req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:15.113927 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] The token cache is being invalidate because project 5579c190401e40e4a47a2540402ac4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:15.114219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:15.114864 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8e36625f-89aa-4b09-9756-1fda7f9c0fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:15.116406 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1353/2733] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:15 2022] DELETE /identity/v3/projects/5579c190401e40e4a47a2540402ac4a3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:15.121293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.121573 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.122102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.143013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:15.144268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:15.184914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:15.204020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1381/2734] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:03:15.212590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.212879 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.213161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.233522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:15.234769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:15.242977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1354/2735] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:15.247366 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.257488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595998-1aa3-4237-94ab-daa4f491e4ad None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.262460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.263084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.263360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.263708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.264353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:03:15.264651 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.266230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.278063 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1382/2736] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 07:03:15.282144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.291686 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da5971-1822-4edc-b339-0a701fb925c1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.296295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.296877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.297154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.297426 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.298083 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.300081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.326141 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1355/2737] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:15 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:15.330259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.340662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb638691-9dd7-454f-bbbd-5317b2f983d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.398852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.399458 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.399738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.400006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.400625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.402537 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.422880 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1383/2738] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:15 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:15.426797 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.436865 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3012ad-09aa-4110-9906-d4b3c9b44f68 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.441475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.442079 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.442359 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.442668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.443319 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.445681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.456228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1356/2739] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:15.460714 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.470658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68131ff2-dd28-4df8-a006-b9f5b16e92f4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.475495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.476080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.476354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.476620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.477280 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.478889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dfd635-2a97-4ef6-b788-ab488c689b2b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.488541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1384/2740] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:15.492587 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.502784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3823f3a-f23e-4d6d-acbd-dc7e7a40d0c2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.507845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.508483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.508760 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.509032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] PATH_INFO: `/v3/projects/2a53bd11138a43238f875a89f85e16bb/users/7125182ef3b5438b99beb6e7a6f7346c/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.523735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a53bd11138a43238f875a89f85e16bb, user_id=7125182ef3b5438b99beb6e7a6f7346c, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.526355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.540067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1357/2741] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:15 2022] PUT /identity/v3/projects/2a53bd11138a43238f875a89f85e16bb/users/7125182ef3b5438b99beb6e7a6f7346c/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:15.551538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.551807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.552081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.579540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] MFA Rules not processed for user `7125182ef3b5438b99beb6e7a6f7346c`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:15.581017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:15.643397 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1385/2742] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:15.654389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:15.656132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:15.711544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5afe7-f5bd-41b1-a5a3-191dd81f5be9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:15.715309 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:15.715906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:15.716184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:15.716453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:15.724660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:15.726065 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741e2d7-a531-4dff-bd76-8522bd609ded service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:15.734717 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1358/2743] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:25.640522 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:25.730855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:35.333152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:35.415120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e7eff-3888-4114-ab3e-8f9e99c5cae3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:35.419797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:35.420423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:35.420703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:35.420975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] PATH_INFO: `/v3/users/7125182ef3b5438b99beb6e7a6f7346c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:35.422102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7125182ef3b5438b99beb6e7a6f7346c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:35.424058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:35.472266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7125182ef3b5438b99beb6e7a6f7346c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:35.479177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7125182ef3b5438b99beb6e7a6f7346c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:35.479797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7125182ef3b5438b99beb6e7a6f7346c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:35.480396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7125182ef3b5438b99beb6e7a6f7346c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:35.493057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7125182ef3b5438b99beb6e7a6f7346c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:35.505141 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1386/2744] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:35 2022] DELETE /identity/v3/users/7125182ef3b5438b99beb6e7a6f7346c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:35.509673 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e375c-3735-4a8e-9896-990230aa453d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:35.510109 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e375c-3735-4a8e-9896-990230aa453d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:35.510199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e375c-3735-4a8e-9896-990230aa453d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:35.528483 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-532e375c-3735-4a8e-9896-990230aa453d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:35.529674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-532e375c-3735-4a8e-9896-990230aa453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:35.586549 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1359/2745] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:35.596146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:35.597541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:35.654244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab796f9-858c-4e71-9ae1-0bda3351c141 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:35.657794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:35.658390 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:35.658697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:35.658963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:35.668122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:35.669543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbd304-1c49-484e-8637-ecce8b20f14a service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:35.679311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1387/2746] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:03:35.949600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e375c-3735-4a8e-9896-990230aa453d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:35.951152 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-532e375c-3735-4a8e-9896-990230aa453d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:35.964729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e375c-3735-4a8e-9896-990230aa453d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:35.970370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:35.972163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:35.972440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:35.972703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] PATH_INFO: `/v3/projects/2a53bd11138a43238f875a89f85e16bb` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:35.974260 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a53bd11138a43238f875a89f85e16bb)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:35.977058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.028814 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a53bd11138a43238f875a89f85e16bb', 'request_id': 'req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:36.034962 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a53bd11138a43238f875a89f85e16bb', 'request_id': 'req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:36.042696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a53bd11138a43238f875a89f85e16bb', 'request_id': 'req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:36.052924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] The token cache is being invalidate because project 2a53bd11138a43238f875a89f85e16bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:36.053208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:36.053826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a0abe3cf-16be-4c8d-b0c1-37a509913ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:36.055333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1360/2747] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:35 2022] DELETE /identity/v3/projects/2a53bd11138a43238f875a89f85e16bb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:36.060547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.060830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.061127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.069006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:36.082684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:36.084371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:36.141102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1388/2748] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:36.149441 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.149724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.149997 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.170176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:36.171392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:36.179132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1361/2749] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:36.183444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.193052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f6474-e8e1-4933-8042-ed08bc550bf8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.198226 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.199125 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.199608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.200089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.200771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:03:36.201071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.202709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.214905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1389/2750] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 07:03:36.219112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.228627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92288-532b-47e7-8cbb-c6fb3da98c91 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.233247 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.233824 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.234098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.234361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.235016 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.237618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.268979 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1362/2751] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:36 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:36.270031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.280361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17211dcd-efde-43b9-afb9-d90114ff5e59 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.337047 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.337667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.337912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.338179 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.338851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.340826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.362258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1390/2752] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:36 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:36.366690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.377705 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b6cb-73e6-488e-b55f-1097ee396efa admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.384667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.385278 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.385551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.385818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.386448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.388058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.399955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1363/2753] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:03:36.404035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.413983 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2c10-9843-44ec-9a8e-6c3397a6f701 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.418518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.419140 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.419428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.419696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.420326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.421918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a7dc8-7687-4c87-a214-a74363143921 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.432266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1391/2754] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:36.436112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.446872 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c547d8-c788-4ab7-9e27-1a4e240b31f0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.451597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.452244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.452528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.452817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] PATH_INFO: `/v3/projects/526610c5c7944c22a4827a1fbfa97443/users/a552060dca064b59ac856298fe11312b/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.468060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526610c5c7944c22a4827a1fbfa97443, user_id=a552060dca064b59ac856298fe11312b, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.470736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.483268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1364/2755] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:36 2022] PUT /identity/v3/projects/526610c5c7944c22a4827a1fbfa97443/users/a552060dca064b59ac856298fe11312b/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:36.494005 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.494303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.494597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.521794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] MFA Rules not processed for user `a552060dca064b59ac856298fe11312b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:36.523027 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:36.579450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:36.589059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1392/2756] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:03:36.596140 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:36.597866 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:36.619933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:36.656476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd28a5c4-ab6b-4a7d-90a1-0a3df49c7d14 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:36.660121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:36.660714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:36.660989 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:36.661256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:36.669511 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:36.670967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0ce1-a808-4f5f-b0d1-e1d865c8ab71 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:36.680179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1365/2757] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:39.689683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:39.753254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96014933-31f0-4ba9-ad62-0bed5e1e2874 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:39.757644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:39.758273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:39.758554 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:39.758855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] PATH_INFO: `/v3/users/a552060dca064b59ac856298fe11312b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:39.759959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a552060dca064b59ac856298fe11312b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:39.761873 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:39.806481 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a552060dca064b59ac856298fe11312b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:39.813588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a552060dca064b59ac856298fe11312b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:39.814221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a552060dca064b59ac856298fe11312b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:39.814832 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a552060dca064b59ac856298fe11312b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:39.826149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a552060dca064b59ac856298fe11312b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:39.838400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1393/2758] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:39 2022] DELETE /identity/v3/users/a552060dca064b59ac856298fe11312b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:39.842959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:39.843414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:39.843497 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:39.862512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:39.863752 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:39.922076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1366/2759] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:39.931757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:39.933165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:39.991321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da465e-eab5-4427-842d-b6c72a6656be admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:39.995096 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:39.995709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:39.996014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:39.996311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.005484 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.007474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4d20de-4ea1-478f-ab06-3b636c70d04e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.019581 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1394/2760] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:40.287440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.289148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:40.305029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38ed8f-4908-43c6-8414-78ec45119e63 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.314795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.315433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.315709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.315982 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] PATH_INFO: `/v3/projects/526610c5c7944c22a4827a1fbfa97443` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.317111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] RBAC: Authorizing `identity:delete_project(project_id=526610c5c7944c22a4827a1fbfa97443)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.319647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.376991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526610c5c7944c22a4827a1fbfa97443', 'request_id': 'req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:40.383844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526610c5c7944c22a4827a1fbfa97443', 'request_id': 'req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:40.392037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526610c5c7944c22a4827a1fbfa97443', 'request_id': 'req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:40.404096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] The token cache is being invalidate because project 526610c5c7944c22a4827a1fbfa97443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:40.404382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:40.405094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52969ac4-3608-4a7c-beb0-3e1dcfa4238f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:40.406609 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1367/2761] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:40 2022] DELETE /identity/v3/projects/526610c5c7944c22a4827a1fbfa97443 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:40.411532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.411848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.412119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.431730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:40.432930 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:40.447753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:40.499075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1395/2762] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:40.507350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.507631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.507903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.527678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:40.528902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:40.537916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1368/2763] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:40.542119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.551963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42437e92-117a-45dd-8917-2f7fd67c180f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.557626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.558259 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.558533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.558831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.559475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:03:40.559773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.561409 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.573772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1396/2764] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:03:40.578045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.587420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9e4da-b9b8-4396-ae94-8e3d403b468d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.592170 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.592871 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.593144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.593492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.594156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.596107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.624471 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1369/2765] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:40.628600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.638605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cdb25-d806-48e8-bf6e-36c47025b373 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.694691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.695376 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.695620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.695887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.696524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.698430 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.719946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1397/2766] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:40.724133 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.734882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e6f2-9982-4cad-bf0f-1fd005b67e3f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.739567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.740742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.741017 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.741284 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.741915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.743528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.753564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1370/2767] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:40.757705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.769392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f989d-4758-4b5d-859f-e72f51cbc41a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.774216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.774924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.775201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.775475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.776109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.777697 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aeb848-14c6-4cba-88fa-136af1196926 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.789492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1398/2768] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:40.793744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:40.804732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af6feb-2f52-40f1-bc58-2d6ea90a92f2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:40.809363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:40.810116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.810389 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.810691 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] PATH_INFO: `/v3/projects/d6ab173a90f94988bc575e802855126f/users/e72783fb3f3d407a8d3b014e646c9728/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.812890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:40.828055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ab173a90f94988bc575e802855126f, user_id=e72783fb3f3d407a8d3b014e646c9728, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:40.830830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:40.843659 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1371/2769] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:40 2022] PUT /identity/v3/projects/d6ab173a90f94988bc575e802855126f/users/e72783fb3f3d407a8d3b014e646c9728/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:40.854919 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:40.855201 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:40.855470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:40.884101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] MFA Rules not processed for user `e72783fb3f3d407a8d3b014e646c9728`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:40.885356 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:40.949486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1399/2770] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:40.953992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.014909 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfc42c-2621-4602-ab94-5c0867d4f57a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.019470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.020064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.020369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.020637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.021312 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.023238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.048694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1372/2771] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:40 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:03:41.053271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.063788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e656d4-e0de-4c81-ac32-b7e037eef849 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.075662 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:41.119757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.120354 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.120636 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.120904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.121827 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.123760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.144293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1400/2772] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:41 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:03:41.148254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.158353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa05612-577b-480d-9dfc-11fe0a8f602c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.162987 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.163568 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.163846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.164116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.164749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.166325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.176783 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1373/2773] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:41.181143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.190848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a0ac-e80e-44b6-9729-bfb261881f9e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.195488 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.196078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.196357 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.196625 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.197261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.198871 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48eca8-a0cc-48cc-bd72-c47d39fa14e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.208478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1401/2774] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:41.212898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.222623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8268c90-d34e-44cc-9e6b-56ba07795f92 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.227827 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.228451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.228729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.228995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] PATH_INFO: `/v3/projects/666a6051696e4fe1b9e34201a0dfd78c/users/8d16d6f766bf445e86485fd3bbf4301e/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.243082 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666a6051696e4fe1b9e34201a0dfd78c, user_id=8d16d6f766bf445e86485fd3bbf4301e, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.245694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.258387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1374/2775] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:41 2022] PUT /identity/v3/projects/666a6051696e4fe1b9e34201a0dfd78c/users/8d16d6f766bf445e86485fd3bbf4301e/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:41.269516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.269798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.270068 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.298514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] MFA Rules not processed for user `8d16d6f766bf445e86485fd3bbf4301e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:41.299837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:41.360989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1402/2776] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:03:41.365625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.428001 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bbbc1-5e44-435e-883d-61643db22876 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.432600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.433190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.433467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.433735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.434372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.436295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.463239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1375/2777] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:03:41 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:03:41.467432 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.478295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a212e-d175-42a5-9ce0-36c2a9ff4889 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.533468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.534070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.534347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.534639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.535288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.537228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.556850 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1403/2778] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:03:41 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:03:41.560725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.571623 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1821b-6041-4bd2-82db-4979dd4167be admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.576593 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.577192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.577468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.577737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.578378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.580076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.591489 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1376/2779] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:41.595755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.606233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cc84b-e22d-4f61-bb4f-7a2de163a731 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.610888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.611534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.611809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.612077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] PATH_INFO: `/v3/projects/452e4b6acbee49128a3467301a49d100/users/b1978f07f9e3411cb0a4686751a562d9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.623860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:41.628171 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452e4b6acbee49128a3467301a49d100, user_id=b1978f07f9e3411cb0a4686751a562d9, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.631990 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.644774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1404/2780] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:41 2022] PUT /identity/v3/projects/452e4b6acbee49128a3467301a49d100/users/b1978f07f9e3411cb0a4686751a562d9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:03:41.649378 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.659282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcfc18-ff25-45ea-b682-cc540164d3e9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.662792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:41.716476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.717066 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.717344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.717614 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.718244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.719874 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff927-2b99-4c8c-aab3-4e63f67e5226 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.729512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1377/2781] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:03:41.733642 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.744368 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b89ba-fc58-4a26-bd94-189c36e50b41 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.749207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.750108 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.750474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.750801 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] PATH_INFO: `/v3/projects/452e4b6acbee49128a3467301a49d100/users/b1978f07f9e3411cb0a4686751a562d9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.753159 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452e4b6acbee49128a3467301a49d100, user_id=b1978f07f9e3411cb0a4686751a562d9, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:41.756813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:41.767805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1405/2782] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:03:41 2022] PUT /identity/v3/projects/452e4b6acbee49128a3467301a49d100/users/b1978f07f9e3411cb0a4686751a562d9/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:41.778122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.778404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.778709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:41.806221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] MFA Rules not processed for user `b1978f07f9e3411cb0a4686751a562d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:41.807543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:41.874074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1378/2783] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:41.885945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:41.887716 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:41.943910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac28e70-d012-41c9-b86a-b9bb25c38519 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:41.947538 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:41.948156 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:41.948457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:41.948724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:42.002442 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:42.003918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:42.013204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1406/2784] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:43.507846 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:43.517280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b4ff4-c5e6-4a9c-bfb1-e42185969ef0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:43.520742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:43.521350 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:43.521621 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:43.521886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:43.575723 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:43.577058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:43.585719 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1379/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:44.108728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:44.117337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b077-116f-43e4-b9d4-87761332dabc service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:44.120987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:44.121859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:44.122135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:44.122404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:44.131295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:44.132706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e20ed-bbc0-4642-8da2-14ee3e40e24e service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:44.141894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1407/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:48.014631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:48.044999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:48.078521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4400157-87ec-419b-9914-45a11a1bcc25 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:48.083849 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:48.084472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:48.084761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:48.085004 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] PATH_INFO: `/v3/users/e72783fb3f3d407a8d3b014e646c9728` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:48.086106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72783fb3f3d407a8d3b014e646c9728)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:48.088014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:48.135045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72783fb3f3d407a8d3b014e646c9728'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.143044 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72783fb3f3d407a8d3b014e646c9728'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.143651 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72783fb3f3d407a8d3b014e646c9728'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.144246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72783fb3f3d407a8d3b014e646c9728'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.151798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72783fb3f3d407a8d3b014e646c9728'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.168480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1380/2787] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:48 2022] DELETE /identity/v3/users/e72783fb3f3d407a8d3b014e646c9728 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:48.173072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:48.173539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:48.173626 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:48.192207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:03:48.193856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:48.250030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1408/2788] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:03:48.260647 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:48.262056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:48.318196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6476784-fbc0-4489-802d-97c1e02579fe admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:48.321848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:48.322440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:48.322755 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:48.323029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:48.331967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:48.333994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:48.343366 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1381/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:03:48.608695 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:48.610139 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:48.622498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5a35d-4d69-4930-9dc0-5ea942f6ef3d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:48.627094 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:48.627849 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:48.628128 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:48.628393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] PATH_INFO: `/v3/projects/d6ab173a90f94988bc575e802855126f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:48.629504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ab173a90f94988bc575e802855126f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:48.632120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:48.644293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:48.679842 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ab173a90f94988bc575e802855126f', 'request_id': 'req-e415b9e6-4732-4725-a802-940ed1519d1c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.685743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ab173a90f94988bc575e802855126f', 'request_id': 'req-e415b9e6-4732-4725-a802-940ed1519d1c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.693727 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ab173a90f94988bc575e802855126f', 'request_id': 'req-e415b9e6-4732-4725-a802-940ed1519d1c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.706939 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] The token cache is being invalidate because project d6ab173a90f94988bc575e802855126f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:48.707227 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.707869 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.709392 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1409/2790] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:48 2022] DELETE /identity/v3/projects/d6ab173a90f94988bc575e802855126f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:03:48.713331 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:48.722912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dbc7d-ab2e-4f2d-892c-efb229f60355 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:48.766193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:48.779943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:48.780561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:48.780832 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:48.781096 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] PATH_INFO: `/v3/users/8d16d6f766bf445e86485fd3bbf4301e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:48.782192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d16d6f766bf445e86485fd3bbf4301e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:48.784117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:48.829823 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d16d6f766bf445e86485fd3bbf4301e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.836968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d16d6f766bf445e86485fd3bbf4301e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.837592 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d16d6f766bf445e86485fd3bbf4301e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.838193 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d16d6f766bf445e86485fd3bbf4301e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.845240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d16d6f766bf445e86485fd3bbf4301e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:48.862975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1382/2791] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:48 2022] DELETE /identity/v3/users/8d16d6f766bf445e86485fd3bbf4301e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:49.139052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:49.140486 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:49.153212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415b9e6-4732-4725-a802-940ed1519d1c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:49.212252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:49.212903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:49.213144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:49.213410 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] PATH_INFO: `/v3/projects/666a6051696e4fe1b9e34201a0dfd78c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:49.214539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] RBAC: Authorizing `identity:delete_project(project_id=666a6051696e4fe1b9e34201a0dfd78c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:49.216493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:49.261192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666a6051696e4fe1b9e34201a0dfd78c', 'request_id': 'req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.266954 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666a6051696e4fe1b9e34201a0dfd78c', 'request_id': 'req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.274446 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666a6051696e4fe1b9e34201a0dfd78c', 'request_id': 'req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.284687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] The token cache is being invalidate because project 666a6051696e4fe1b9e34201a0dfd78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:49.284977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.285595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.287647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1410/2792] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:49 2022] DELETE /identity/v3/projects/666a6051696e4fe1b9e34201a0dfd78c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:49.291668 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:49.293057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:49.305656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59fe6-248d-4101-87dd-23975e0f274a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:49.360076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:49.360688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:49.360961 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:49.361227 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] PATH_INFO: `/v3/users/b1978f07f9e3411cb0a4686751a562d9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:49.362325 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1978f07f9e3411cb0a4686751a562d9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:49.364249 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:49.409789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1978f07f9e3411cb0a4686751a562d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.417443 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1978f07f9e3411cb0a4686751a562d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.418050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1978f07f9e3411cb0a4686751a562d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.418685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1978f07f9e3411cb0a4686751a562d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.425679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1978f07f9e3411cb0a4686751a562d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.430666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:49.445385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1383/2793] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:03:49 2022] DELETE /identity/v3/users/b1978f07f9e3411cb0a4686751a562d9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:03:49.723833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:03:49.725553 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:03:49.738441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7b570-f1a2-40ac-bf5b-f1f3c554d66a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:03:49.769014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:49.799807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:03:49.800367 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:03:49.800644 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:03:49.800909 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] PATH_INFO: `/v3/projects/452e4b6acbee49128a3467301a49d100` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:03:49.802041 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=452e4b6acbee49128a3467301a49d100)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:03:49.803968 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:03:49.849904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452e4b6acbee49128a3467301a49d100', 'request_id': 'req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.855759 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452e4b6acbee49128a3467301a49d100', 'request_id': 'req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.863007 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452e4b6acbee49128a3467301a49d100', 'request_id': 'req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.873498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] The token cache is being invalidate because project 452e4b6acbee49128a3467301a49d100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:03:49.873785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.874407 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ddcfa14-795b-46a7-8d83-91dbfa354ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:03:49.875932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1411/2794] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:03:49 2022] DELETE /identity/v3/projects/452e4b6acbee49128a3467301a49d100 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:03:59.441931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=5,keystone:DELETE=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:03:59.870229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:10.160697 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:04:10.162690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:10.218826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39b887-1f49-4475-9306-a1a02863f4d8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:04:10.264784 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:04:10.265369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:04:10.265642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:04:10.265907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:04:10.267617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:10.336725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:04:10.338150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:04:10.347060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1384/2795] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:04:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:04:20.342886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=73 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:42.062133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e404478-073d-4487-8083-17e695b98b70 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:04:42.062655 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e404478-073d-4487-8083-17e695b98b70 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:04:42.062747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e404478-073d-4487-8083-17e695b98b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:04:42.081595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1e404478-073d-4487-8083-17e695b98b70 None None] MFA Rules not processed for user `b997560479c642e0ac43c0b08a1bf157`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:04:42.082885 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1e404478-073d-4487-8083-17e695b98b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:42.137417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1412/2796] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:04:42.456272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:04:42.457666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:42.512795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493736b2-3124-4858-95f5-01620fb5e436 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:04:42.516369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:04:42.516959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:04:42.517234 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:04:42.517501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:04:42.519208 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:42.528931 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:04:42.530329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:04:42.538771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1385/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:04:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:04:45.680187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e404478-073d-4487-8083-17e695b98b70 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:04:45.681807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1e404478-073d-4487-8083-17e695b98b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:45.761193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e404478-073d-4487-8083-17e695b98b70 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:04:45.766708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:04:45.767320 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:04:45.767595 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:04:45.767863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:04:45.769599 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:45.782621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:04:45.784213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:04:45.794789 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1413/2798] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:04:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:04:52.534633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=26 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:55.790306 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=59 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:04:56.740154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:04:56.742218 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:56.763058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147e5-36fe-4d7c-bb8a-2e6f2ccb748b service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:04:56.766656 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:04:56.767240 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:04:56.767515 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:04:56.767782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:04:56.769440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:04:56.778980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:04:56.780395 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:04:56.788476 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1386/2799] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:06.784818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=10 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:21.358014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:21.367182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b035-c264-4595-b926-eab007658d69 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:21.371016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:21.371600 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:21.371872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:21.372137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:21.373844 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:21.435709 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:21.437166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:21.447009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1414/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:24.542696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:24.544179 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:24.562540 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571b72-6bf3-40c8-889d-442bd3acbc7b service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:24.567166 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:24.568115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:24.568608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:24.568897 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] PATH_INFO: `/v3/users/d85160b01d824bc3a32b425fab98d2c7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:24.570703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85160b01d824bc3a32b425fab98d2c7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:24.572582 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:24.623878 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85160b01d824bc3a32b425fab98d2c7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.632836 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85160b01d824bc3a32b425fab98d2c7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.633512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85160b01d824bc3a32b425fab98d2c7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.634142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85160b01d824bc3a32b425fab98d2c7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.641576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-50076001-2c1c-4d67-984e-cd252a6ab1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85160b01d824bc3a32b425fab98d2c7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.662907 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1387/2801] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:24 2022] DELETE /identity/v3/users/d85160b01d824bc3a32b425fab98d2c7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:24.835699 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:24.846295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb0edd-78ba-41e6-b70e-46772864c676 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:24.903617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:24.904249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:24.904530 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:24.904798 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] PATH_INFO: `/v3/projects/0c5d2a3f2d944529b8c505fa23b5a10f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:24.905910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5d2a3f2d944529b8c505fa23b5a10f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:24.907851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:24.952372 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5d2a3f2d944529b8c505fa23b5a10f', 'request_id': 'req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.962167 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5d2a3f2d944529b8c505fa23b5a10f', 'request_id': 'req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.971371 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5d2a3f2d944529b8c505fa23b5a10f', 'request_id': 'req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.983113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] The token cache is being invalidate because project 0c5d2a3f2d944529b8c505fa23b5a10f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:24.983405 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.984028 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9fcff32b-1ff0-4dd0-8908-e5b1b12e0b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:24.985978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1415/2802] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:24 2022] DELETE /identity/v3/projects/0c5d2a3f2d944529b8c505fa23b5a10f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:24.993465 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:24.993890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:24.994304 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.021910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:25.024117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:25.082300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1388/2803] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:25.090208 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.090495 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.090830 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.110169 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:25.111415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:25.120143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1416/2804] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:25.124226 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.134245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13beaeba-7f8f-44c4-9b73-76005ece6de9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.138739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.139326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.139600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.139867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.140501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:25.140798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.142369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.153528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1389/2805] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:25.157659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.166833 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b0bc9-4392-49d3-af02-9d1e26baeaca None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.171469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.172113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.172391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.172663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.173303 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.175237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.190870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:25.205105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1417/2806] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:25.209060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.219194 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962bd029-1a3c-43c8-92e2-2eb9070a5b6b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.274093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.274636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.274917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.275188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.275817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.277721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.285536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:25.302459 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1390/2807] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/users => generated 278 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:25.307805 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.321422 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1585e7-e8ec-49b7-8170-6be113347533 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.326747 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.327360 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.327652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.327914 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.328558 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.330162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.341202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1418/2808] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:25.345270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.355161 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a366cc-159c-4b4f-8914-4b741cf32224 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.359757 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.360346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.360625 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.360894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.361530 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.363142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9db42-b8c7-473c-8271-a9c4a16f2b62 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.374222 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1391/2809] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:25.378339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.388788 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e443ed-02a3-4064-b6e2-cbc9e9f1fbf2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.393429 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.394066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.394345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.394669 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] PATH_INFO: `/v3/projects/9e9d364adb86493d88930ec79e983dba/users/bc506902f300477bb3d47340105db594/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.408857 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9d364adb86493d88930ec79e983dba, user_id=bc506902f300477bb3d47340105db594, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.411541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.424109 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1419/2810] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:25 2022] PUT /identity/v3/projects/9e9d364adb86493d88930ec79e983dba/users/bc506902f300477bb3d47340105db594/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:25.436219 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.436508 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.436785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.465892 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] MFA Rules not processed for user `bc506902f300477bb3d47340105db594`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:25.467132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:25.529863 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1392/2811] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:25.534361 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.597177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a93028-c216-4e05-a52d-32a596743f73 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.602736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.603362 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.603641 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.603913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.604551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.606474 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.639929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1420/2812] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:25.644034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.658513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77000a-8700-4340-914b-78526ea56b92 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.730679 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.731286 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.731560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.731828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.732464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.734357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.756213 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1393/2813] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:25.760521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.771342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff266b-a49d-4dc1-a449-120f9c8dc379 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.776014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.776608 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.776884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.777152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.777785 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.779403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.790683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1421/2814] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:25.794788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.805357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8debc6-f56b-47ed-b22f-ee96c50dd5b7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.809922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.810613 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.810891 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.811159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] PATH_INFO: `/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d/users/bd4cdd0b570b49c48694a4ba81aa7a7c/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.825412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abc1beb6b8c40c3bbf6dbd41bf0850d, user_id=bd4cdd0b570b49c48694a4ba81aa7a7c, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.828062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.841047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1394/2815] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:25 2022] PUT /identity/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d/users/bd4cdd0b570b49c48694a4ba81aa7a7c/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:25.844903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.855770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3f94c-484f-4375-ba50-56730b90cc51 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.912104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.912707 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.913601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.913875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.914516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.916212 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebaa00-db0a-463a-bf88-7da732104362 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.929142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1422/2816] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:05:25.934175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:25.944667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78872b2c-aa6c-4016-938c-f0e4857435bd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:25.949334 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:25.949978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.950253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.950527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] PATH_INFO: `/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d/users/bd4cdd0b570b49c48694a4ba81aa7a7c/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:25.952473 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abc1beb6b8c40c3bbf6dbd41bf0850d, user_id=bd4cdd0b570b49c48694a4ba81aa7a7c, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:25.955129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:25.965813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1395/2817] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:25 2022] PUT /identity/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d/users/bd4cdd0b570b49c48694a4ba81aa7a7c/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:25.976612 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:25.976927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:25.977204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:26.004959 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] MFA Rules not processed for user `bd4cdd0b570b49c48694a4ba81aa7a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:26.006181 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:26.024863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:26.079560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1423/2818] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:26.091254 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:26.092985 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:26.115785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:26.152637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1bea8-25bf-407d-a3a1-22ec93bd9ec4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:26.156205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:26.156795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:26.157068 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:26.157333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:26.166689 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:26.168105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:26.177320 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1396/2819] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:26.251681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:26.262067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060c684-e984-4e77-a7fd-656ba5f7405e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:26.265596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:26.266182 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:26.266524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:26.266816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:26.322568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:26.323978 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be363251-3434-47f3-a7ec-5bcf8f5b8f27 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:26.333002 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1424/2820] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:36.173270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=17 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:36.328810 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=48 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:41.075457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:41.137289 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c498cd-3ca9-42fa-bcb8-e500036b8888 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:41.142265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:41.143215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:41.143712 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:41.144123 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] PATH_INFO: `/v3/users/bc506902f300477bb3d47340105db594` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:41.145246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc506902f300477bb3d47340105db594)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:41.147244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:41.196336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc506902f300477bb3d47340105db594'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.203820 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc506902f300477bb3d47340105db594'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.204745 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc506902f300477bb3d47340105db594'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.205381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc506902f300477bb3d47340105db594'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.212488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc506902f300477bb3d47340105db594'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.232370 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1397/2821] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:41 2022] DELETE /identity/v3/users/bc506902f300477bb3d47340105db594 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:41.237074 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:41.237564 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:41.237652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:41.261237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:41.262469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:41.322814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1425/2822] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:41.332769 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:41.334239 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:41.391485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e8cd1-d1b3-4688-b6d1-add0d91d0f8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:41.395147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:41.395744 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:41.396019 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:41.396288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:41.405654 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:41.407122 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:41.417268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1398/2823] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:05:41.691774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:41.693631 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:41.706268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d50657-711a-4c7e-beec-ac1581858a39 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:41.711992 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:41.712738 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:41.713015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:41.713285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] PATH_INFO: `/v3/projects/9e9d364adb86493d88930ec79e983dba` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:41.714460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9d364adb86493d88930ec79e983dba)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:41.716541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:41.769775 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9d364adb86493d88930ec79e983dba', 'request_id': 'req-3a2d76e9-5eba-425a-b3e5-d662648acd67'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.777556 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9d364adb86493d88930ec79e983dba', 'request_id': 'req-3a2d76e9-5eba-425a-b3e5-d662648acd67'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.785373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9d364adb86493d88930ec79e983dba', 'request_id': 'req-3a2d76e9-5eba-425a-b3e5-d662648acd67'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.797358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] The token cache is being invalidate because project 9e9d364adb86493d88930ec79e983dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:41.797758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.798415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.799910 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1426/2824] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:41 2022] DELETE /identity/v3/projects/9e9d364adb86493d88930ec79e983dba => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:41.804021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:41.813598 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd7854-75ff-469f-badd-a3db16317005 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:41.832163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:41.872025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:41.872640 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:41.872913 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:41.873180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] PATH_INFO: `/v3/users/bd4cdd0b570b49c48694a4ba81aa7a7c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:41.874272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd4cdd0b570b49c48694a4ba81aa7a7c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:41.876205 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:41.921855 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd4cdd0b570b49c48694a4ba81aa7a7c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.928753 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd4cdd0b570b49c48694a4ba81aa7a7c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.929369 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd4cdd0b570b49c48694a4ba81aa7a7c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.929976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd4cdd0b570b49c48694a4ba81aa7a7c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.937594 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1e727ce3-82ed-47de-8c55-ab8881936d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd4cdd0b570b49c48694a4ba81aa7a7c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:41.955990 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1399/2825] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:41 2022] DELETE /identity/v3/users/bd4cdd0b570b49c48694a4ba81aa7a7c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:42.222195 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.223665 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:42.239947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d76e9-5eba-425a-b3e5-d662648acd67 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.297614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.298249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.298533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.298836 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] PATH_INFO: `/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.299957 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abc1beb6b8c40c3bbf6dbd41bf0850d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.301868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.333912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:42.351151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abc1beb6b8c40c3bbf6dbd41bf0850d', 'request_id': 'req-0e94c697-17be-4eb6-9472-bdadc86eab80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:42.357120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abc1beb6b8c40c3bbf6dbd41bf0850d', 'request_id': 'req-0e94c697-17be-4eb6-9472-bdadc86eab80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:42.364088 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abc1beb6b8c40c3bbf6dbd41bf0850d', 'request_id': 'req-0e94c697-17be-4eb6-9472-bdadc86eab80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:42.375282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] The token cache is being invalidate because project 2abc1beb6b8c40c3bbf6dbd41bf0850d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:42.375569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:42.376188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0e94c697-17be-4eb6-9472-bdadc86eab80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:42.377693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1427/2826] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:42 2022] DELETE /identity/v3/projects/2abc1beb6b8c40c3bbf6dbd41bf0850d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:42.383413 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.384000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.384000 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.403300 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:42.404504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:42.462084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1400/2827] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:42.469814 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.470098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.470370 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.489703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:42.491439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:42.499594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1428/2828] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:42.503664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.514192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e5794-ecf5-4b37-8aa1-465105e963bf None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.518749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.519329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.519606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.519877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.520507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:42.520804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.522435 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.533579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1401/2829] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 07:05:42.537594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.546660 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f48c6c-d986-4826-ac96-9ec9dc2fbe7d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.551224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.551813 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.552089 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.552358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.553480 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.555457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.585780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1429/2830] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:42.589974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.599950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b49a4b-0ec4-4fc4-9f37-fbc02d420c4c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.626144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:42.662116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.662740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.663013 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.663281 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.663896 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.665795 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.686220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1402/2831] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:42.690363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.701034 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4dc89-557f-49f2-b4c0-38619eedaed4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.705938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.706539 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.706864 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.707137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.707782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.709403 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.719283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1430/2832] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:42.723024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.733360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99edbe-376e-406c-987a-261128fb52b9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.737895 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.738474 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.738779 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.739050 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.739680 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.741870 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a925d569-7083-470f-b655-6030c298dcae admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.752610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1403/2833] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:42.756571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.766571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b694-6bf5-40c2-8eef-ff1c09cb89ea admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.771265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.771895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.772168 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.772434 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] PATH_INFO: `/v3/projects/5e6ce1e172e34a4f9b8bb35309c1e75c/users/b3a8171871ad4c6d9efb5f6a7fc9a054/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.786445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6ce1e172e34a4f9b8bb35309c1e75c, user_id=b3a8171871ad4c6d9efb5f6a7fc9a054, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.789584 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:42.802521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1431/2834] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:42 2022] PUT /identity/v3/projects/5e6ce1e172e34a4f9b8bb35309c1e75c/users/b3a8171871ad4c6d9efb5f6a7fc9a054/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:42.814282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.814562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.814856 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.842811 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] MFA Rules not processed for user `b3a8171871ad4c6d9efb5f6a7fc9a054`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:42.844030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:42.906115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1404/2835] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:42.910705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:42.972809 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128dbee1-9b86-4465-ad9c-ff064d933a35 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:42.978060 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:42.978690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:42.978974 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:42.979246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:42.979897 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:42.981800 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.009395 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1432/2836] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:42 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:43.013661 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.024302 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d75d28-cb0c-450b-8f4f-a52177e50383 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.079965 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.080554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.080833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.081102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.081731 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.084228 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.105513 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1405/2837] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:43 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:43.109647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.119920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a76f3-eb9d-4aac-90ab-7e937c696989 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.124487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.125072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.125347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.125614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.126247 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.127876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.138203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1433/2838] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:43.142059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.152415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e551-2540-4fb4-a28e-766b5c07dc98 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.157054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.157631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.157908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.158183 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] PATH_INFO: `/v3/projects/666905ddd14b4f51b4ab4d4616496d9e/users/ad1fb66a845d41d5b65275c95c84fe84/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.173097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=666905ddd14b4f51b4ab4d4616496d9e, user_id=ad1fb66a845d41d5b65275c95c84fe84, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.175767 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.187917 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1406/2839] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:43 2022] PUT /identity/v3/projects/666905ddd14b4f51b4ab4d4616496d9e/users/ad1fb66a845d41d5b65275c95c84fe84/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:43.191723 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.201646 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0ef2-ae0b-4444-a192-51125681a0ed admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.222078 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:43.260445 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.261032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.261304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.261571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.262192 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.263945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf06b19-35e4-406c-9c56-74d9996ffd1e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.273339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1434/2840] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:43.277544 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.283093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:43.289458 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdb0ee-b5cf-4f10-9f75-da77a63aaa5e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.294466 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.295134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.295412 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.295682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] PATH_INFO: `/v3/projects/666905ddd14b4f51b4ab4d4616496d9e/users/ad1fb66a845d41d5b65275c95c84fe84/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.297635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=666905ddd14b4f51b4ab4d4616496d9e, user_id=ad1fb66a845d41d5b65275c95c84fe84, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.300297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.310960 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1407/2841] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:43 2022] PUT /identity/v3/projects/666905ddd14b4f51b4ab4d4616496d9e/users/ad1fb66a845d41d5b65275c95c84fe84/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:43.321549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.321831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.322106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.349860 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] MFA Rules not processed for user `ad1fb66a845d41d5b65275c95c84fe84`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:43.351105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:43.418103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1435/2842] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:05:43.430774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.432527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:43.489537 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285daec5-8104-460a-ad09-72d36acecbdd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.493810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.494696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.494978 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.495251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.504137 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.505610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.515028 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1408/2843] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:43.836856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.846101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fbb08e-3ea6-4379-abc4-63ec10b2539b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.851032 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad1fb66a845d41d5b65275c95c84fe84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '666905ddd14b4f51b4ab4d4616496d9e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.851696 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.852000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.852278 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.852924 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:43.853229 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.854846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.866868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1436/2844] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:43.871126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.880253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21329662-54ee-47af-90e9-85b64671faad service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.885467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad1fb66a845d41d5b65275c95c84fe84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '666905ddd14b4f51b4ab4d4616496d9e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.886053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.886340 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.886641 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.887279 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.889167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.915139 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1409/2845] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:43 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:43.919117 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:43.929379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c29f32-5ecd-4116-9d73-9d88eee93025 tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:43.983233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad1fb66a845d41d5b65275c95c84fe84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '666905ddd14b4f51b4ab4d4616496d9e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:43.983826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:43.984110 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:43.984387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:43.985084 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:43.985369 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:43.987077 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:43.999543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1437/2846] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:44.046554 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:44.057033 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9dedb-43a6-40ab-99d5-0e978479b6bf tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:44.061502 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad1fb66a845d41d5b65275c95c84fe84', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '666905ddd14b4f51b4ab4d4616496d9e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:44.062107 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:44.062385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:44.062685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] PATH_INFO: `/v3/projects/535d6761592d4525aa31ce697f8d9944` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:44.063813 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorizing `identity:delete_project(project_id=535d6761592d4525aa31ce697f8d9944)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:44.065682 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:44.109902 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535d6761592d4525aa31ce697f8d9944', 'request_id': 'req-583aec80-16e9-436f-bcba-c25f231cd34d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.116903 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535d6761592d4525aa31ce697f8d9944', 'request_id': 'req-583aec80-16e9-436f-bcba-c25f231cd34d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.124200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535d6761592d4525aa31ce697f8d9944', 'request_id': 'req-583aec80-16e9-436f-bcba-c25f231cd34d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.134695 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] The token cache is being invalidate because project 535d6761592d4525aa31ce697f8d9944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:44.134981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.135608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-583aec80-16e9-436f-bcba-c25f231cd34d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.137042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1410/2847] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:44 2022] DELETE /identity/v3/projects/535d6761592d4525aa31ce697f8d9944 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:44.336918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:44.338783 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:44.370105 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:44.405199 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a705d0-1108-4eb9-89cf-56efec3edd9d tempest-VolumeQuotaClassesTest-1915487732 tempest-VolumeQuotaClassesTest-1915487732-project-admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:44.469288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:44.469942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:44.470224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:44.470524 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] PATH_INFO: `/v3/users/b3a8171871ad4c6d9efb5f6a7fc9a054` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:44.471749 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a8171871ad4c6d9efb5f6a7fc9a054)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:44.473717 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:44.523059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a8171871ad4c6d9efb5f6a7fc9a054'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.530894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a8171871ad4c6d9efb5f6a7fc9a054'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.531899 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a8171871ad4c6d9efb5f6a7fc9a054'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.532562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a8171871ad4c6d9efb5f6a7fc9a054'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.544970 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a8171871ad4c6d9efb5f6a7fc9a054'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:44.557213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1438/2848] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:44 2022] DELETE /identity/v3/users/b3a8171871ad4c6d9efb5f6a7fc9a054 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:44.561664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:44.562118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:44.562118 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:44.580915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:44.582103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:44.627720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:44.641373 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1411/2849] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:44.651705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:44.653255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:44.709137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb118f9c-f840-4512-a4d7-46146872eeb0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:44.712824 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:44.713426 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:44.713703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:44.713975 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:44.720766 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:44.723265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:44.725773 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:44.736818 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1439/2850] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:05:44.998670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.000223 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:45.014265 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2152a-9ec4-44bb-badd-690a057ad456 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:45.019175 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:45.019798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.020071 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.020337 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] PATH_INFO: `/v3/projects/5e6ce1e172e34a4f9b8bb35309c1e75c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.022169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6ce1e172e34a4f9b8bb35309c1e75c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:45.025163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:45.070782 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6ce1e172e34a4f9b8bb35309c1e75c', 'request_id': 'req-c522c532-4e7b-45c5-ad18-420b4d1b1575'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.076694 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6ce1e172e34a4f9b8bb35309c1e75c', 'request_id': 'req-c522c532-4e7b-45c5-ad18-420b4d1b1575'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.084634 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6ce1e172e34a4f9b8bb35309c1e75c', 'request_id': 'req-c522c532-4e7b-45c5-ad18-420b4d1b1575'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.094992 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] The token cache is being invalidate because project 5e6ce1e172e34a4f9b8bb35309c1e75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:45.095330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.095973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.097550 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1412/2851] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:44 2022] DELETE /identity/v3/projects/5e6ce1e172e34a4f9b8bb35309c1e75c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:45.101516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.111079 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd04ae4-5ede-4a64-b2d9-5e15fadaaf44 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:45.165506 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:45.166124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.166404 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.166704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] PATH_INFO: `/v3/users/ad1fb66a845d41d5b65275c95c84fe84` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.167807 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1fb66a845d41d5b65275c95c84fe84)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:45.169703 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:45.216013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1fb66a845d41d5b65275c95c84fe84'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.222882 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1fb66a845d41d5b65275c95c84fe84'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.223497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1fb66a845d41d5b65275c95c84fe84'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.224144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1fb66a845d41d5b65275c95c84fe84'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.236010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c9382a5-392e-423c-94fc-8ce31b880fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1fb66a845d41d5b65275c95c84fe84'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.248526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1440/2852] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:45 2022] DELETE /identity/v3/users/ad1fb66a845d41d5b65275c95c84fe84 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:45.518225 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.519677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:45.533248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522c532-4e7b-45c5-ad18-420b4d1b1575 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:45.589501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:45.590150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.590391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.590688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] PATH_INFO: `/v3/projects/666905ddd14b4f51b4ab4d4616496d9e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.591798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=666905ddd14b4f51b4ab4d4616496d9e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:45.593690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:45.639741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666905ddd14b4f51b4ab4d4616496d9e', 'request_id': 'req-0156ba76-7570-4bf2-9986-4f14dd9864a9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.645749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666905ddd14b4f51b4ab4d4616496d9e', 'request_id': 'req-0156ba76-7570-4bf2-9986-4f14dd9864a9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.653886 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666905ddd14b4f51b4ab4d4616496d9e', 'request_id': 'req-0156ba76-7570-4bf2-9986-4f14dd9864a9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.664282 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] The token cache is being invalidate because project 666905ddd14b4f51b4ab4d4616496d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:45.664581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.665196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0156ba76-7570-4bf2-9986-4f14dd9864a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:45.666670 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1413/2853] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:45 2022] DELETE /identity/v3/projects/666905ddd14b4f51b4ab4d4616496d9e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:45.671411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.671851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.671935 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.691511 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:45.692724 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:45.749663 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1441/2854] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:45.812922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.813199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.813467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.828106 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:45.834932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:45.836648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:45.844431 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1414/2855] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:45.848591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.858238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb765-e5ed-4d49-8180-7f7eb27335ce None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:45.862740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:45.863325 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.863598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.863859 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.864750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:45.865076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:45.867008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:45.878328 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1442/2856] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:45.882996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.893674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2171c1-2bbd-4ca2-ade6-57cffb13de28 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:45.898973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:45.899567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:45.899956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:45.900242 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:45.900995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:45.902922 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:45.930173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1415/2857] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:45 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:45.934387 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:45.939650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:45.946423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bcc0b-f33e-4fb8-b133-776a1c8dcadb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.002635 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.003248 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.003526 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.003797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.004457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.006900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.031091 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1443/2858] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:45 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:46.032190 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.042975 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8684a-f968-4031-b7b4-efe9b5b45d4d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.047953 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.048571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.048845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.049111 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.049743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.051362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.061764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1416/2859] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:05:46.065780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.076075 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf4e5-4814-40d1-a558-e5c6c2e9dc62 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.080653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.081234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.081509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.081774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.082398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.084038 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1a037-7e96-4742-9469-721cf6568e67 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.094549 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1444/2860] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:46.098561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.108463 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb552-2eab-445b-b3d6-fb9543449666 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.113074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.113732 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.113995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.114267 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] PATH_INFO: `/v3/projects/21714d813e9240e7ab5aaeccd59cff0c/users/6a14488bdb4a4caa8a3991b0b045a81c/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.129385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21714d813e9240e7ab5aaeccd59cff0c, user_id=6a14488bdb4a4caa8a3991b0b045a81c, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.132663 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.145606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1417/2861] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:46 2022] PUT /identity/v3/projects/21714d813e9240e7ab5aaeccd59cff0c/users/6a14488bdb4a4caa8a3991b0b045a81c/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:46.155489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.155770 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.156037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.183170 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] MFA Rules not processed for user `6a14488bdb4a4caa8a3991b0b045a81c`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:46.184373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:46.248659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1445/2862] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:46.253144 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.313789 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75db8-5759-489d-a60f-adbe3d5db719 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.318543 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.319156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.319436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.319707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.320330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.322221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.348069 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1418/2863] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:46 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:46.352797 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.363503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1dabe-7e69-4413-b26f-e52d9df04af5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.423122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.423726 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.424001 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.424267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.424887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.426820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.442385 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:46.448872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1446/2864] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:46 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:46.452794 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.466792 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31adb8-373d-4f80-98a6-1d5a9e47219c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.471505 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.472131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.472410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.472684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.473329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.475053 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.485796 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1419/2865] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:46.490673 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.500846 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9fa4fb-ae1d-4f6d-bdfe-a72e0a4c09ed admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.505925 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.506569 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.506872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.507142 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] PATH_INFO: `/v3/projects/557568a15d034571b40d5e23ef266d4a/users/93b52036d8634cde96d5aa383f1c5895/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.522286 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557568a15d034571b40d5e23ef266d4a, user_id=93b52036d8634cde96d5aa383f1c5895, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.525119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.539548 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1447/2866] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:46 2022] PUT /identity/v3/projects/557568a15d034571b40d5e23ef266d4a/users/93b52036d8634cde96d5aa383f1c5895/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:46.543422 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.553381 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b32ad9-85e5-44f1-b3c3-505232ecbd81 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.609472 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.610113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.610336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.610633 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.611272 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.612857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82f847-e574-4a65-acae-b8b79a9ff68b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.623148 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1420/2867] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:46.627250 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.638129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e5c67-ab42-4200-94c3-ebcdff450fd7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.642741 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.643381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.643658 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.643927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] PATH_INFO: `/v3/projects/557568a15d034571b40d5e23ef266d4a/users/93b52036d8634cde96d5aa383f1c5895/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.645829 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] RBAC: Authorizing `identity:create_grant(project_id=557568a15d034571b40d5e23ef266d4a, user_id=93b52036d8634cde96d5aa383f1c5895, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.649207 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.659396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1448/2868] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:46 2022] PUT /identity/v3/projects/557568a15d034571b40d5e23ef266d4a/users/93b52036d8634cde96d5aa383f1c5895/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:46.671912 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.672196 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.672493 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.697993 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:46.701407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] MFA Rules not processed for user `93b52036d8634cde96d5aa383f1c5895`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:46.703322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:46.772200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1421/2869] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:46.783295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.785013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:46.841562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab9061-ab25-40ec-8f3d-ea9183f7455c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.845254 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.845851 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.846126 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.846391 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.855867 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.857312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:46.870119 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1449/2870] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:46.920237 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:46.921615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:46.934185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef216fbd-8a3d-4bfe-878d-afe86b6ca898 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:46.937737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:46.938322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:46.938618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:46.938889 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:46.993905 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:46.995534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735c4f3-a5fe-4b5f-829f-b2180bd5f7e3 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:47.004353 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1422/2871] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:50.288263 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:50.349623 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab76f2-e6be-407b-a518-04fe3b1a74c3 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:50.354112 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:50.354768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:50.355052 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:50.355324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] PATH_INFO: `/v3/users/6a14488bdb4a4caa8a3991b0b045a81c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:50.356427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a14488bdb4a4caa8a3991b0b045a81c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:50.358334 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:50.404649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a14488bdb4a4caa8a3991b0b045a81c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.411388 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a14488bdb4a4caa8a3991b0b045a81c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.412551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a14488bdb4a4caa8a3991b0b045a81c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.413152 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a14488bdb4a4caa8a3991b0b045a81c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.423498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:50.426010 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a14488bdb4a4caa8a3991b0b045a81c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.440058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1450/2872] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:50 2022] DELETE /identity/v3/users/6a14488bdb4a4caa8a3991b0b045a81c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:50.444798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:50.445275 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:50.445382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:50.464470 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:50.465683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:50.523743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1423/2873] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:50.533593 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:50.535015 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:50.591144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b92e8-17fc-4da6-9c38-d2c4d84eff88 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:50.594780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:50.595382 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:50.595650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:50.595913 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:50.605253 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:50.606702 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:50.615998 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1451/2874] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:50.887195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:50.888617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:50.905147 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a1cc5-7395-4c99-9c2d-f745cfce91c4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:50.910025 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:50.910693 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:50.911037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:50.911388 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] PATH_INFO: `/v3/projects/21714d813e9240e7ab5aaeccd59cff0c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:50.913059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21714d813e9240e7ab5aaeccd59cff0c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:50.915059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:50.950956 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:50.966715 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21714d813e9240e7ab5aaeccd59cff0c', 'request_id': 'req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.973501 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21714d813e9240e7ab5aaeccd59cff0c', 'request_id': 'req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.980972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21714d813e9240e7ab5aaeccd59cff0c', 'request_id': 'req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.991182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] The token cache is being invalidate because project 21714d813e9240e7ab5aaeccd59cff0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:50.991464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.992076 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:50.994538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1424/2875] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:50 2022] DELETE /identity/v3/projects/21714d813e9240e7ab5aaeccd59cff0c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:50.998750 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.008761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e345c0-3fd1-4f69-b8eb-e3e9c436c430 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.064471 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.065215 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.065560 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.065900 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] PATH_INFO: `/v3/users/93b52036d8634cde96d5aa383f1c5895` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.067149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b52036d8634cde96d5aa383f1c5895)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.069618 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.118308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b52036d8634cde96d5aa383f1c5895'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.126462 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b52036d8634cde96d5aa383f1c5895'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.127205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b52036d8634cde96d5aa383f1c5895'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.127819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b52036d8634cde96d5aa383f1c5895'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.141711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e21f0fc7-9d5d-4426-bef5-fcef63c007bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b52036d8634cde96d5aa383f1c5895'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.157675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1452/2876] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:50 2022] DELETE /identity/v3/users/93b52036d8634cde96d5aa383f1c5895 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:51.435981 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.437433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:51.452509 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cbde9-d6c0-43f2-8e4b-ebf4b03030c5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.511890 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.512512 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.512790 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.513060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] PATH_INFO: `/v3/projects/557568a15d034571b40d5e23ef266d4a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.514220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] RBAC: Authorizing `identity:delete_project(project_id=557568a15d034571b40d5e23ef266d4a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.516145 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.562411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557568a15d034571b40d5e23ef266d4a', 'request_id': 'req-225e0324-4e33-4fe9-b016-ab0eb547863a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.568375 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557568a15d034571b40d5e23ef266d4a', 'request_id': 'req-225e0324-4e33-4fe9-b016-ab0eb547863a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.576538 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557568a15d034571b40d5e23ef266d4a', 'request_id': 'req-225e0324-4e33-4fe9-b016-ab0eb547863a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.587336 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] The token cache is being invalidate because project 557568a15d034571b40d5e23ef266d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:51.587622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.588245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-225e0324-4e33-4fe9-b016-ab0eb547863a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:51.589761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1425/2877] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:51 2022] DELETE /identity/v3/projects/557568a15d034571b40d5e23ef266d4a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:51.594808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.595294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.595380 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.609491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:51.616076 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:51.617794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:51.675106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1453/2878] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:51.683988 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.684283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.684557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.704500 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:51.705714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:51.714074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1426/2879] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:51.718217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.727866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6687d14-1220-4710-a014-268cc0b38733 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.732427 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.733021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.733301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.733574 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.734217 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:51.734517 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.736143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.747620 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1454/2880] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:51.751885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.762414 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18119261-9aab-4ccd-9029-862df55f5ce5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.767092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.767631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.767906 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.768172 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.768801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.770762 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.797294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1427/2881] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:51 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:51.801639 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.812601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf5ca-75f8-4e13-a672-c556ea3f8360 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.867337 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.867949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.868225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.868496 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.869381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.871335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.892318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1455/2882] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:51 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:51.897436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.907739 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc0d69-9289-412f-80dc-09bc6d843a70 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.912720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.913305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.913583 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.913862 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.914488 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.916093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.925822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1428/2883] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:51.930230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.940166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45011b3f-4ed9-4074-a86f-157be4acba0f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.944736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.945315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.945591 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.945876 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.946507 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.948145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5324b-9139-4032-8ca9-81b208f5a09b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:51.958423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1456/2884] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:51.962382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:51.972399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58279-06c4-402b-885a-b37c73a70f5f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:51.977430 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:51.978084 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:51.978345 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:51.978646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] PATH_INFO: `/v3/projects/c96008a1cd9f45aba1f229f8179837da/users/d3b4c46053934a8ba0aac0a349794c99/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:51.986768 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:51.996364 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96008a1cd9f45aba1f229f8179837da, user_id=d3b4c46053934a8ba0aac0a349794c99, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:51.999180 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.012105 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1429/2885] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:51 2022] PUT /identity/v3/projects/c96008a1cd9f45aba1f229f8179837da/users/d3b4c46053934a8ba0aac0a349794c99/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:52.022415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.022705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.022973 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.050279 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] MFA Rules not processed for user `d3b4c46053934a8ba0aac0a349794c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:52.051542 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:52.114416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1457/2886] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:52.119236 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.178536 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90438242-78f9-45fe-9c49-43a51f740127 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.183030 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.183610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.183884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.184149 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.184761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.186667 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.212475 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1430/2887] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:52 2022] POST /identity/v3/projects => generated 389 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:52.216533 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.222042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:52.229674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8b946-0bd6-401a-85e4-da1d70aeed7f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.284080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.284614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.284888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.285155 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.285780 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.288343 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.309071 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1458/2888] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:52 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:52.313059 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.323399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b9d9b-4332-4486-9655-09a834b1f7d9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.327995 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.328579 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.328882 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.329150 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.329786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.331399 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.342462 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1431/2889] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:05:52.346348 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.357045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5226dc1-34e1-4d11-a22f-6f1415a2c907 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.361656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.362300 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.362597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.362893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] PATH_INFO: `/v3/projects/235222f7bac44bdc8063f61f2e350e5d/users/ae7e75b16e4840c49bb88f345aec2dbe/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.377381 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] RBAC: Authorizing `identity:create_grant(project_id=235222f7bac44bdc8063f61f2e350e5d, user_id=ae7e75b16e4840c49bb88f345aec2dbe, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.380145 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.392933 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1459/2890] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:52 2022] PUT /identity/v3/projects/235222f7bac44bdc8063f61f2e350e5d/users/ae7e75b16e4840c49bb88f345aec2dbe/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:52.397428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.407821 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb427a8-d2e1-4549-9a9b-065e53ebcd61 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.463734 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.464346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.464603 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.464873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.465510 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.467125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca636112-cab2-4554-8ea1-b44276081720 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.476875 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1432/2891] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:52.480987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.491240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1995e-7adb-472c-b430-7d8431f8336e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.496035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.496667 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.496942 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.497211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] PATH_INFO: `/v3/projects/235222f7bac44bdc8063f61f2e350e5d/users/ae7e75b16e4840c49bb88f345aec2dbe/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.499127 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=235222f7bac44bdc8063f61f2e350e5d, user_id=ae7e75b16e4840c49bb88f345aec2dbe, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.501731 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.512118 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1460/2892] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:52 2022] PUT /identity/v3/projects/235222f7bac44bdc8063f61f2e350e5d/users/ae7e75b16e4840c49bb88f345aec2dbe/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:52.523885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.524214 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.524561 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.558198 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] MFA Rules not processed for user `ae7e75b16e4840c49bb88f345aec2dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:52.560092 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:52.609563 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:52.634788 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1433/2893] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:52.645460 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.647870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:52.704518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2623a-3749-4781-9637-72019c859e45 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.708160 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.709319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.709617 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.709856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.721273 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.722794 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b1bd0-8c84-4ad0-8ef4-b72195401365 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.732123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1461/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:52.863948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:52.938894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaca7a1-055c-483a-a179-c3c6f31eadd2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:52.944058 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:52.944677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:52.944950 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:52.945216 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] PATH_INFO: `/v3/users/d3b4c46053934a8ba0aac0a349794c99` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:52.946318 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b4c46053934a8ba0aac0a349794c99)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:52.948261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:52.994506 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b4c46053934a8ba0aac0a349794c99'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.002095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b4c46053934a8ba0aac0a349794c99'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.002763 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b4c46053934a8ba0aac0a349794c99'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.003379 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b4c46053934a8ba0aac0a349794c99'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.010976 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b4c46053934a8ba0aac0a349794c99'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.027702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1434/2895] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:52 2022] DELETE /identity/v3/users/d3b4c46053934a8ba0aac0a349794c99 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:53.032177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:53.032470 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:53.032744 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:53.051151 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:53.052887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:53.101239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:53.111684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1462/2896] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:05:53.122188 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:53.123646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:53.181200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2099f1-3256-48ca-935b-bc3e013a3d57 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:53.184815 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:53.185407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:53.185683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:53.185952 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:53.195718 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:53.197134 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:53.206677 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1435/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:53.465905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:53.467364 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:53.481489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c11e3-60e3-404a-8a44-0f5afeb411d9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:53.486516 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:53.487191 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:53.487468 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:53.487736 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] PATH_INFO: `/v3/projects/c96008a1cd9f45aba1f229f8179837da` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:53.488856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96008a1cd9f45aba1f229f8179837da)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:53.492035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:53.538345 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96008a1cd9f45aba1f229f8179837da', 'request_id': 'req-688b7285-837f-41ed-8fd3-fb45eb68df83'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.544509 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96008a1cd9f45aba1f229f8179837da', 'request_id': 'req-688b7285-837f-41ed-8fd3-fb45eb68df83'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.551255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96008a1cd9f45aba1f229f8179837da', 'request_id': 'req-688b7285-837f-41ed-8fd3-fb45eb68df83'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.562198 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] The token cache is being invalidate because project c96008a1cd9f45aba1f229f8179837da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:53.562482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.563113 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.564588 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1463/2898] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:53 2022] DELETE /identity/v3/projects/c96008a1cd9f45aba1f229f8179837da => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:53.568722 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:53.574853 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:53.578740 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4edb25-418e-496c-a29d-16ffc49c03a5 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:53.637297 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:53.637845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:53.638169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:53.638438 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] PATH_INFO: `/v3/users/ae7e75b16e4840c49bb88f345aec2dbe` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:53.639581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7e75b16e4840c49bb88f345aec2dbe)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:53.641481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:53.685761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7e75b16e4840c49bb88f345aec2dbe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.693338 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7e75b16e4840c49bb88f345aec2dbe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.694094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7e75b16e4840c49bb88f345aec2dbe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.694943 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7e75b16e4840c49bb88f345aec2dbe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.701973 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c1e2af8e-176e-4dcf-aefb-b9925521227e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7e75b16e4840c49bb88f345aec2dbe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:53.719121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1436/2899] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:53 2022] DELETE /identity/v3/users/ae7e75b16e4840c49bb88f345aec2dbe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:53.983597 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:53.985048 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:53.998308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b7285-837f-41ed-8fd3-fb45eb68df83 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.056691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.057400 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.057679 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.057966 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] PATH_INFO: `/v3/projects/235222f7bac44bdc8063f61f2e350e5d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.059100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=235222f7bac44bdc8063f61f2e350e5d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.061003 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.106257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235222f7bac44bdc8063f61f2e350e5d', 'request_id': 'req-5d787f43-d269-4fe1-901e-7ea66c95f8b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:54.112203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235222f7bac44bdc8063f61f2e350e5d', 'request_id': 'req-5d787f43-d269-4fe1-901e-7ea66c95f8b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:54.119435 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235222f7bac44bdc8063f61f2e350e5d', 'request_id': 'req-5d787f43-d269-4fe1-901e-7ea66c95f8b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:54.129691 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] The token cache is being invalidate because project 235222f7bac44bdc8063f61f2e350e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:54.130494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:54.131162 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5d787f43-d269-4fe1-901e-7ea66c95f8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:54.132630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1464/2900] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:53 2022] DELETE /identity/v3/projects/235222f7bac44bdc8063f61f2e350e5d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:54.137404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.137685 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.138098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.157709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:54.158996 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:54.215936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1437/2901] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:54.223575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.223863 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.224135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.243238 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:54.244692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:54.252466 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1465/2902] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:54.257207 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.267131 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2752b-e0e1-4d45-b1f5-238d3823daa5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.272192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.272806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.273078 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.273344 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.273971 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:54.274264 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.275881 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.287080 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1438/2903] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:54.291449 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.296977 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:54.300837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7ab68-0f7c-495c-9468-c82791f329c5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.307681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.308284 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.308746 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.309177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.309816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.311758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.339528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1466/2904] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:54.343584 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.353522 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656381a5-deb4-4c68-872a-ab088b385316 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.361169 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:54.412898 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.413420 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.413696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.413964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.414596 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.416700 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.437606 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1439/2905] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:54.441893 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.452632 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2773d-ba6b-4355-a5d0-a3b154c84149 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.457243 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.457856 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.458143 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.458415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.459124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.460722 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.470711 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1467/2906] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:54.474660 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.484612 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357b0a4-7fde-4e4c-8085-07d1e7527c96 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.489958 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.490548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.490845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.491114 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.491756 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.493360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368810df-ce98-44ff-9f01-ffe118b5ed3d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.504158 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1440/2907] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:54.509529 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.519519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba54719-d4db-45c0-83b6-898f45c01e63 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.524124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.524774 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.525054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.525326 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] PATH_INFO: `/v3/projects/3a57786a8e1d4311b1642ece1725abcd/users/52bec557d20141328ef1ad2c76cc52b2/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.539423 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a57786a8e1d4311b1642ece1725abcd, user_id=52bec557d20141328ef1ad2c76cc52b2, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.542059 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.555023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1468/2908] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:54 2022] PUT /identity/v3/projects/3a57786a8e1d4311b1642ece1725abcd/users/52bec557d20141328ef1ad2c76cc52b2/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:54.565047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.565360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.565629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.593407 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] MFA Rules not processed for user `52bec557d20141328ef1ad2c76cc52b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:54.594646 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:54.658590 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1441/2909] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:54.663123 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.727342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953ef1f-5242-4824-9f60-bcfe192e6eb5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.731985 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.732585 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.732866 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.733137 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.733761 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.735708 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.762401 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1469/2910] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:54.766524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.776999 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9f467-a7cc-4e89-9bd3-2e7df2bd9efe None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.831771 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.832360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.832630 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.832924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.833608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.835532 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.856322 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1442/2911] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:54 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:54.860331 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.871342 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa447ed0-e78e-44e4-8da2-04f2cb8a4a9a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.875892 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.876478 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.876752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.877020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.877652 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.879283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.889534 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1470/2912] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:54.893560 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.904021 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3e0fa-688d-48ab-b6b7-fdcb3e2c1c05 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:54.908622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:54.909283 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:54.909557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:54.909826 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] PATH_INFO: `/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c/users/1564777a7c2e4acfbcd2f0304e488865/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:54.913417 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:54.926185 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eeca8346f0840a2aba2a35f0ab4bb9c, user_id=1564777a7c2e4acfbcd2f0304e488865, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:54.929427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:54.942244 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1443/2913] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:54 2022] PUT /identity/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c/users/1564777a7c2e4acfbcd2f0304e488865/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:54.946315 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:54.956249 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3a90c-ad4f-47ba-8eae-2f68a5c365a4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:55.020081 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:55.020763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:55.020964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:55.021232 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:55.021870 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:55.023512 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba50e6a-c8bf-46db-b529-2459db3bcfd9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:55.033087 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1471/2914] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:55.037972 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:55.048270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce204f-3e1c-43da-9d6b-1b6c98574b2a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:55.053455 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:55.054087 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:55.054358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:55.054664 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] PATH_INFO: `/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c/users/1564777a7c2e4acfbcd2f0304e488865/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:55.056542 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eeca8346f0840a2aba2a35f0ab4bb9c, user_id=1564777a7c2e4acfbcd2f0304e488865, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:55.059195 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:55.069504 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1444/2915] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:55 2022] PUT /identity/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c/users/1564777a7c2e4acfbcd2f0304e488865/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:55.080728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:55.081016 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:55.081292 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:55.109359 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] MFA Rules not processed for user `1564777a7c2e4acfbcd2f0304e488865`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:55.110606 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:55.115550 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:55.178102 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1472/2916] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:55.189437 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:55.191160 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:55.248047 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73259b2a-9ed1-4622-80b8-1e5ab96b164e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:55.251737 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:55.252333 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:55.252610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:55.252880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:55.307308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:55.308730 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a3245-c281-46f9-ae15-138866fb9404 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:55.317451 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1445/2917] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:05:57.868615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:57.933373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3a344-cf7c-4193-a32b-767f5f16e6a0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:57.937834 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:57.938489 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:57.938796 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:57.939067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] PATH_INFO: `/v3/users/52bec557d20141328ef1ad2c76cc52b2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:57.940209 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bec557d20141328ef1ad2c76cc52b2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:57.942120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:57.988272 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bec557d20141328ef1ad2c76cc52b2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:57.996333 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bec557d20141328ef1ad2c76cc52b2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:57.996944 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bec557d20141328ef1ad2c76cc52b2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:57.997543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bec557d20141328ef1ad2c76cc52b2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.009594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bec557d20141328ef1ad2c76cc52b2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.021433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1473/2918] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:57 2022] DELETE /identity/v3/users/52bec557d20141328ef1ad2c76cc52b2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:05:58.026054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:58.026529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:58.026649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:58.046308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:58.047567 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:58.104845 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1446/2919] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:58.114843 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:58.116266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:58.142353 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:58.174737 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0690b8-a6ff-4048-989b-b632e4499aa8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:58.178312 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:58.178934 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:58.179210 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:58.179475 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:58.188196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:58.189609 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:58.200946 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1474/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:05:58.469129 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:58.470674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:58.483221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba7207-bc3d-4bb9-bfc0-605564c45c54 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:58.487720 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:58.488342 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:58.488615 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:58.488884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] PATH_INFO: `/v3/projects/3a57786a8e1d4311b1642ece1725abcd` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:58.490031 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a57786a8e1d4311b1642ece1725abcd)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:58.492014 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:58.516967 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:58.540293 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a57786a8e1d4311b1642ece1725abcd', 'request_id': 'req-d23c91ee-9754-4d54-8b9e-089013fa4241'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.546182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a57786a8e1d4311b1642ece1725abcd', 'request_id': 'req-d23c91ee-9754-4d54-8b9e-089013fa4241'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.553432 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a57786a8e1d4311b1642ece1725abcd', 'request_id': 'req-d23c91ee-9754-4d54-8b9e-089013fa4241'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.563597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] The token cache is being invalidate because project 3a57786a8e1d4311b1642ece1725abcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:58.563876 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.564484 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.565939 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1447/2921] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:58 2022] DELETE /identity/v3/projects/3a57786a8e1d4311b1642ece1725abcd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:58.570568 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:58.580180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d36219-a488-4109-a46b-d72643c2449b service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:58.636180 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:58.636730 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:58.637004 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:58.637269 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] PATH_INFO: `/v3/users/1564777a7c2e4acfbcd2f0304e488865` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:58.638373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1564777a7c2e4acfbcd2f0304e488865)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:58.640308 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:58.685293 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1564777a7c2e4acfbcd2f0304e488865'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.692971 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1564777a7c2e4acfbcd2f0304e488865'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.693605 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1564777a7c2e4acfbcd2f0304e488865'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.694203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1564777a7c2e4acfbcd2f0304e488865'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.706255 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-08109e3a-077d-4e33-b831-c5b15db42d76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1564777a7c2e4acfbcd2f0304e488865'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:58.718624 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1475/2922] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:05:58 2022] DELETE /identity/v3/users/1564777a7c2e4acfbcd2f0304e488865 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:58.991060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:58.992491 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:59.007423 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c91ee-9754-4d54-8b9e-089013fa4241 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.063027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.063648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.063926 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.064199 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] PATH_INFO: `/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.065301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eeca8346f0840a2aba2a35f0ab4bb9c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.067243 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.111857 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eeca8346f0840a2aba2a35f0ab4bb9c', 'request_id': 'req-e32cb916-7b00-4f0b-90d6-dd450914dbd0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:59.118352 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eeca8346f0840a2aba2a35f0ab4bb9c', 'request_id': 'req-e32cb916-7b00-4f0b-90d6-dd450914dbd0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:59.125894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eeca8346f0840a2aba2a35f0ab4bb9c', 'request_id': 'req-e32cb916-7b00-4f0b-90d6-dd450914dbd0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:59.136901 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] The token cache is being invalidate because project 8eeca8346f0840a2aba2a35f0ab4bb9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:05:59.137189 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:59.137810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e32cb916-7b00-4f0b-90d6-dd450914dbd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:05:59.139326 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1448/2923] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:05:58 2022] DELETE /identity/v3/projects/8eeca8346f0840a2aba2a35f0ab4bb9c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:59.144133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.144575 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.144657 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.163725 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:59.164932 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:59.204515 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:59.225596 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1476/2924] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:05:59.233358 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.233653 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.233924 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.253798 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:59.255055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:59.262825 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1449/2925] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:59.266905 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.276531 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab927fe-68a5-4b84-956e-ab962334f5b2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.281062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.281653 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.281929 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.282205 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.282887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:05:59.283187 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.284784 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.296498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1477/2926] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:05:59.300569 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.310372 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400966b-7765-4036-aa74-85453a7d79aa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.315097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.315681 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.315959 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.316233 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.316854 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.318780 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.344777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1450/2927] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:59.349285 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.359135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020b02b-d4fd-4238-aa43-1cc20a275ddd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.415682 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.416283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.416561 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.416831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.417465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.419415 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.440211 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1478/2928] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:59.444392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.455121 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874fe70-13d0-4884-950c-a2e721a8ee1b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.459759 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.460357 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.460635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.460908 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.461548 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.463177 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.468642 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:59.474348 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1451/2929] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:59.478482 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.488589 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ceb8c4-114b-4011-a2bf-779cbf2e9d9e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.494070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.494694 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.494972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.495239 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.495872 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.497457 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bcf69-6a93-44de-b6c6-ff342ee14aa9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.507163 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1479/2930] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 18 07:05:59.511376 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.521095 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265e0d3-cb36-4f92-92b1-5ea00fe89ab5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.525721 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.526360 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.526671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.526932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] PATH_INFO: `/v3/projects/ff378d6654574e339e96f03d3c5baa40/users/afb5717804bb4eed9c002e4a95177b15/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.541525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff378d6654574e339e96f03d3c5baa40, user_id=afb5717804bb4eed9c002e4a95177b15, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.544167 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.556932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1452/2931] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:59 2022] PUT /identity/v3/projects/ff378d6654574e339e96f03d3c5baa40/users/afb5717804bb4eed9c002e4a95177b15/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:59.566947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.567265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.567546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.596877 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] MFA Rules not processed for user `afb5717804bb4eed9c002e4a95177b15`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:05:59.598135 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:05:59.660129 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1480/2932] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:05:59.665154 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.725343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f9bca-e545-4226-b6b4-3e0d0d2a0dbe admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.729933 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.730516 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.730822 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.731093 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.731710 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.733601 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.759268 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1453/2933] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:05:59.763304 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.774433 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c98d4c-b92a-43ee-9839-3741c3d902b2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.811879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:05:59.830958 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.831562 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.831841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.832111 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.832758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.835202 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.855989 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1481/2934] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:05:59.859867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.871765 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6751f-2103-4858-8cf6-fd5303466e05 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.876346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.876936 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.877212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.877480 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.878115 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.879777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.890162 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1454/2935] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:59.894045 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.905035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b50169-547a-42e3-97ce-a7db14dd877a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.909868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.910455 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.910758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.911630 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.912270 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.913908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9ecfe-4401-42e0-ba8d-73cdf3f05637 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.924120 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1482/2936] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:05:59.928248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:05:59.938224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea6fca-5929-4721-832c-ce976401cc5c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:05:59.943355 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:05:59.943994 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.944274 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.944547 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] PATH_INFO: `/v3/projects/a93bf47c17d54f4f9c2f4de4ca7fa52c/users/4dbb095547314bf0844d93130c8abf06/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:05:59.959622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93bf47c17d54f4f9c2f4de4ca7fa52c, user_id=4dbb095547314bf0844d93130c8abf06, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:05:59.962308 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:05:59.975436 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1455/2937] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:05:59 2022] PUT /identity/v3/projects/a93bf47c17d54f4f9c2f4de4ca7fa52c/users/4dbb095547314bf0844d93130c8abf06/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:05:59.986777 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:05:59.987058 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:05:59.987324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.015713 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] MFA Rules not processed for user `4dbb095547314bf0844d93130c8abf06`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:00.016908 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:00.077916 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1483/2938] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:05:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 18 07:06:00.082410 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.144270 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a92e438-af3e-4c29-8260-6fa6f684c47b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.149529 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.150112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.150387 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.150678 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.151301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.153363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.178385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1456/2939] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:06:00 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:06:00.182339 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.193257 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f75e7-7080-4328-93db-d5ec2fa1c78b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.246852 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.247379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.247654 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.247920 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.248541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.250441 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.273144 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1484/2940] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:06:00 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:06:00.277094 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.287295 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4584a-8b90-44c4-97eb-1667a3d8875d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.291867 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.292464 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.292741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.293011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.293652 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.295276 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.300392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:00.307362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1457/2941] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:06:00.311216 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.321559 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77138bb-e374-4304-b328-1fc9bd662b9c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.326101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.326748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.327025 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.327296 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] PATH_INFO: `/v3/projects/7a125e25258c40e989500e90bfa95600/users/aa962481a72d4d558831203d77073462/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.342233 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a125e25258c40e989500e90bfa95600, user_id=aa962481a72d4d558831203d77073462, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.344938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.357592 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1485/2942] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:06:00 2022] PUT /identity/v3/projects/7a125e25258c40e989500e90bfa95600/users/aa962481a72d4d558831203d77073462/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:00.361405 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.372184 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbd594-3014-4e70-a016-a2ff38620f21 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.429229 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.429828 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.430110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.430385 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.431056 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.432636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82127e74-45df-48db-8a59-f526a00c3ea8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.442073 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1458/2943] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:06:00.446672 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.457193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5ace0-8b0a-4833-98c8-93f51fb7c32c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.462335 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.462994 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.463274 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.463541 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] PATH_INFO: `/v3/projects/7a125e25258c40e989500e90bfa95600/users/aa962481a72d4d558831203d77073462/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.465536 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a125e25258c40e989500e90bfa95600, user_id=aa962481a72d4d558831203d77073462, role_id=e1b7d4bac8bf43a9a2483cbdd3620ef2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.468107 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.474951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:00.480165 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1486/2944] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:06:00 2022] PUT /identity/v3/projects/7a125e25258c40e989500e90bfa95600/users/aa962481a72d4d558831203d77073462/roles/e1b7d4bac8bf43a9a2483cbdd3620ef2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:06:00.490288 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.490616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.490880 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.518503 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] MFA Rules not processed for user `aa962481a72d4d558831203d77073462`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:00.519690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:00.586674 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1459/2945] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:00.597915 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:00.601042 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:00.660621 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8a07-0683-41fd-bb42-70c120ad965b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:00.664888 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:00.665499 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:00.665778 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:00.666054 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:00.723964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:00.725411 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:00.734477 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1487/2946] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:02.250156 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:02.251600 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:02.263253 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e209b45-744d-45c8-a49e-99f2b21d44ca None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:02.266766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:02.267346 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:02.267619 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:02.267887 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:02.322074 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:02.323448 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:02.332032 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1460/2947] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:02.847615 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:02.856868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd69e0-5076-4204-ab4a-ad837df83a75 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:02.860440 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:02.861035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:02.861311 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:02.861578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:02.870647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:02.872066 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435762c-11be-4e2e-b929-51f94d7daabd service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:02.881213 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1488/2948] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:06.750102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:06.771037 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:06.812051 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaddbaa-66a6-4fc8-866a-a5f0ae5c26e4 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:06.816446 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:06.817055 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:06.817329 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:06.817597 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] PATH_INFO: `/v3/users/afb5717804bb4eed9c002e4a95177b15` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:06.818703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb5717804bb4eed9c002e4a95177b15)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:06.821081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:06.869774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb5717804bb4eed9c002e4a95177b15'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:06.877269 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb5717804bb4eed9c002e4a95177b15'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:06.877873 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb5717804bb4eed9c002e4a95177b15'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:06.878468 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb5717804bb4eed9c002e4a95177b15'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:06.885829 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb5717804bb4eed9c002e4a95177b15'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:06.903102 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1461/2949] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:06:06 2022] DELETE /identity/v3/users/afb5717804bb4eed9c002e4a95177b15 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:06.907704 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:06.908203 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:06.908309 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:06.927053 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:06.928268 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:06.990305 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1489/2950] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:07.000525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:07.001928 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:07.060324 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b31c8-d04a-4199-b041-474c552f0377 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:07.063948 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:07.064564 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:07.064841 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:07.065113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:07.074608 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:07.076024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:07.085629 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1462/2951] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:07.355176 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:07.356705 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:07.370743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b1248-76d3-4a3c-99bc-fd52d25d1a32 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:07.376173 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:07.376819 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:07.377095 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:07.377363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] PATH_INFO: `/v3/projects/ff378d6654574e339e96f03d3c5baa40` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:07.378497 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff378d6654574e339e96f03d3c5baa40)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:07.380453 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:07.423674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:07.434772 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff378d6654574e339e96f03d3c5baa40', 'request_id': 'req-af9334b1-d138-428a-9237-748e2944b98c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.441941 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff378d6654574e339e96f03d3c5baa40', 'request_id': 'req-af9334b1-d138-428a-9237-748e2944b98c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.450425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff378d6654574e339e96f03d3c5baa40', 'request_id': 'req-af9334b1-d138-428a-9237-748e2944b98c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.462252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] The token cache is being invalidate because project ff378d6654574e339e96f03d3c5baa40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:06:07.462546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.463258 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.465017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1490/2952] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:06:07 2022] DELETE /identity/v3/projects/ff378d6654574e339e96f03d3c5baa40 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:07.468786 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:07.479428 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d313-59e1-47ee-9e76-f95aeec43993 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:07.522211 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:07.540804 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:07.541427 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:07.541707 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:07.541980 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] PATH_INFO: `/v3/users/4dbb095547314bf0844d93130c8abf06` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:07.543125 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbb095547314bf0844d93130c8abf06)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:07.545034 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:07.589837 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbb095547314bf0844d93130c8abf06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.597116 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbb095547314bf0844d93130c8abf06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.597724 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbb095547314bf0844d93130c8abf06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.598323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbb095547314bf0844d93130c8abf06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.605703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbb095547314bf0844d93130c8abf06'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:07.623675 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1463/2953] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:06:07 2022] DELETE /identity/v3/users/4dbb095547314bf0844d93130c8abf06 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:07.889751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:07.891234 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:07.907551 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9334b1-d138-428a-9237-748e2944b98c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:07.965124 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:07.965760 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:07.966006 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:07.966271 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] PATH_INFO: `/v3/projects/a93bf47c17d54f4f9c2f4de4ca7fa52c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:07.967396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93bf47c17d54f4f9c2f4de4ca7fa52c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:07.969288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.015062 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93bf47c17d54f4f9c2f4de4ca7fa52c', 'request_id': 'req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.021106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93bf47c17d54f4f9c2f4de4ca7fa52c', 'request_id': 'req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.028614 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93bf47c17d54f4f9c2f4de4ca7fa52c', 'request_id': 'req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.039465 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] The token cache is being invalidate because project a93bf47c17d54f4f9c2f4de4ca7fa52c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:06:08.039748 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.040378 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.041841 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1491/2954] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:06:07 2022] DELETE /identity/v3/projects/a93bf47c17d54f4f9c2f4de4ca7fa52c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:06:08.045802 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.047221 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:08.059888 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cff4c-a5a6-4e6c-9d75-b30104b84a19 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.114940 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.115555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.115833 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.116103 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] PATH_INFO: `/v3/users/aa962481a72d4d558831203d77073462` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.117212 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa962481a72d4d558831203d77073462)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.119404 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.165391 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa962481a72d4d558831203d77073462'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.172525 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa962481a72d4d558831203d77073462'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.173138 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa962481a72d4d558831203d77073462'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.173743 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa962481a72d4d558831203d77073462'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.180617 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b242dede-6a81-4d18-a368-049194ed6c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa962481a72d4d558831203d77073462'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.181714 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:08.200110 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1464/2955] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:06:08 2022] DELETE /identity/v3/users/aa962481a72d4d558831203d77073462 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:08.464346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.465902 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:08.480492 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabf9f0-970e-4649-9e4b-a8804f3c06eb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.532098 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:08.539771 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.540421 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.540912 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.541358 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] PATH_INFO: `/v3/projects/7a125e25258c40e989500e90bfa95600` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.542656 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a125e25258c40e989500e90bfa95600)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.544498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.588728 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a125e25258c40e989500e90bfa95600', 'request_id': 'req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.594650 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a125e25258c40e989500e90bfa95600', 'request_id': 'req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.601927 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a125e25258c40e989500e90bfa95600', 'request_id': 'req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.612219 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] The token cache is being invalidate because project 7a125e25258c40e989500e90bfa95600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:06:08.612521 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.613133 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e1ef0c74-7b4e-45bd-8643-cd0324b96d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:08.614686 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1492/2956] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:06:08 2022] DELETE /identity/v3/projects/7a125e25258c40e989500e90bfa95600 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:08.620294 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.620595 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.620894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.640671 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:08.641938 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:08.698711 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1465/2957] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:08.707753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.708037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.708321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.727674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:08.728889 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:08.737266 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1493/2958] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:08.741246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.751692 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039545b-3de5-4f3b-b90e-c2eff5d1a28e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.756261 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.756848 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.757124 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.757946 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.758599 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:06:08.758911 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.760486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.771645 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1466/2959] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 18 07:06:08.775858 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.785261 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee92c6-830f-46a2-b9ce-272b7653c3d2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.789903 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.790493 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.790802 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.791072 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.791710 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.793692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.821428 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1494/2960] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:06:08 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:06:08.825637 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.835527 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1106a2-14a2-43c3-9274-33b6c7c90f6e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.891045 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.891635 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.891914 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.892187 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.892801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.894729 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.916403 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1467/2961] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:06:08 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 18 07:06:08.920503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.931294 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852028e6-d6f0-418f-adb2-af7dbc92fc37 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.935831 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.936417 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.936693 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.936963 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.937594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.939224 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.948837 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1495/2962] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:06:08.952557 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.962551 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fefa62-d450-4aa7-a1a8-a4ebb6fbfeee admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:08.967126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:08.967703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:08.967977 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:08.968246 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:08.968877 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:08.970449 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788aa2a0-127f-4b97-a992-ed0fabdf466d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:08.981224 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1468/2963] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:06:08.985104 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:08.996023 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114692b-b92d-484b-8660-4f979bd35ddb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:09.000677 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:09.001321 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:09.001601 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:09.001875 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] PATH_INFO: `/v3/projects/523fb59392af43bcb2be01e17406cef7/users/9842848655fe479bbea42fd0f2ca4ffa/roles/d7498f66178947778670d4d051df8536` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:09.016057 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=523fb59392af43bcb2be01e17406cef7, user_id=9842848655fe479bbea42fd0f2ca4ffa, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:09.018743 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:09.031246 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1496/2964] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:06:08 2022] PUT /identity/v3/projects/523fb59392af43bcb2be01e17406cef7/users/9842848655fe479bbea42fd0f2ca4ffa/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:09.036132 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0be89-a34f-4d55-b36c-961d02338980 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:09.036440 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0be89-a34f-4d55-b36c-961d02338980 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:09.036709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0be89-a34f-4d55-b36c-961d02338980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:09.056349 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-89f0be89-a34f-4d55-b36c-961d02338980 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:09.057492 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-89f0be89-a34f-4d55-b36c-961d02338980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:09.096810 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:09.116486 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1469/2965] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:09.127000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:09.128690 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:09.183965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd26f2b-3205-4e12-9871-775e98406f6e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:09.187552 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:09.188146 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:09.188425 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:09.188698 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:09.197528 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:09.198987 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:09.208504 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1497/2966] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:09.823029 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e411-daf9-4b9e-91ff-d242a533ed53 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:09.823305 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e411-daf9-4b9e-91ff-d242a533ed53 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:09.823575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e411-daf9-4b9e-91ff-d242a533ed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:09.851963 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8d25e411-daf9-4b9e-91ff-d242a533ed53 None None] MFA Rules not processed for user `9842848655fe479bbea42fd0f2ca4ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:09.853394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8d25e411-daf9-4b9e-91ff-d242a533ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:09.921843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1470/2967] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:09.932578 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:09.934067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:09.963051 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:09.995469 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b89c-7bdd-40bd-80dc-123e4090aab6 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:09.999346 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:09.999949 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:10.000222 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:10.000487 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:10.009532 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:10.011080 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:10.019721 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1498/2968] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:11.370526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:11.371067 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:11.371159 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:11.390761 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] MFA Rules not processed for user `9842848655fe479bbea42fd0f2ca4ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:11.391883 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:11.399562 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1471/2969] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:14.533986 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:14.535692 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:14.614000 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1311237-16e3-461c-a950-28f4568308e4 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:14.621031 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:14.621647 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:14.621926 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:14.622197 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:14.624070 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:14.637590 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:14.639230 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c197a4-b5be-4ef6-999b-0eec0ce27d25 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:14.649898 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1499/2970] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:18.983764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:18.999884 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00a2f-6487-4171-92d2-5c6aa2f6630d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:19.006481 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04ddfa59d1834ba3a7f1f27e963b090e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:19.007807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:19.008088 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:19.008361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:19.010245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:19.027113 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:19.029363 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:19.049749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1472/2971] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:24.332149 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6944b-ba3f-4308-9c21-07105ae5e5a1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:24.332637 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6944b-ba3f-4308-9c21-07105ae5e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:24.332806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6944b-ba3f-4308-9c21-07105ae5e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:24.351879 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9fd6944b-ba3f-4308-9c21-07105ae5e5a1 None None] MFA Rules not processed for user `9842848655fe479bbea42fd0f2ca4ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:24.353648 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9fd6944b-ba3f-4308-9c21-07105ae5e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:24.360413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1500/2972] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:06:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:27.380382 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:27.390571 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ec24c-e579-431e-8eee-42afc4d6ecf3 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:27.394844 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:27.395627 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:27.395910 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:27.396182 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:27.398176 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:27.483040 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:27.484526 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:27.495415 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1473/2973] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:34.346678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=52 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:37.490764 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=75 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:43.848347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:43.848806 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:43.848904 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:43.870826 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:06:43.872035 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:43.927103 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1501/2974] 10.222.0.20 () {58 vars in 1009 bytes} [Sun Sep 18 07:06:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:06:43.972575 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:43.974209 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:44.045696 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a09a-550f-4ab9-a303-664060fbab6b service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:44.050785 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:44.051392 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:44.051666 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:44.051932 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:44.060453 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:44.062091 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:44.071248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1474/2975] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:06:44.248204 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:44.260282 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e4a61-cfd6-4e72-ad5f-d22ba5479476 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:44.264803 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:44.265683 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:44.266030 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:44.266373 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:44.268265 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:44.282894 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:44.284816 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:44.299101 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1502/2976] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:06:47.979622 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:47.981925 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:48.112916 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d659-4735-4e59-a165-027c54702681 service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:48.120559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:48.120559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:48.120559 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:48.121861 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:48.124749 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:48.137929 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:48.139499 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:48.152572 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1475/2977] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:06:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 18 07:06:52.076808 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:52.078384 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:52.095301 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83742eca-da9a-41ee-8ab9-153ce5b64253 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:52.100188 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:52.100675 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:52.100952 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:52.101267 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] PATH_INFO: `/v3/users/b997560479c642e0ac43c0b08a1bf157` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:52.102416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b997560479c642e0ac43c0b08a1bf157)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:52.104363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:52.150100 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b997560479c642e0ac43c0b08a1bf157'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.157350 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b997560479c642e0ac43c0b08a1bf157'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.157955 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b997560479c642e0ac43c0b08a1bf157'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.158565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b997560479c642e0ac43c0b08a1bf157'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.170240 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b997560479c642e0ac43c0b08a1bf157'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.182413 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1503/2978] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:06:52 2022] DELETE /identity/v3/users/b997560479c642e0ac43c0b08a1bf157 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:52.358081 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:52.378248 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2817a04-7074-4b75-9ca1-6b874d455a34 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:52.456807 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:52.457742 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:52.458048 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:52.458323 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] PATH_INFO: `/v3/projects/f707372e1ec948dcb7e89b9a046cccfc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:52.466991 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f707372e1ec948dcb7e89b9a046cccfc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:52.468974 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:52.501683 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:52.533657 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f707372e1ec948dcb7e89b9a046cccfc', 'request_id': 'req-10db2e20-5529-402f-8460-8e8782e59a44'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.543885 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f707372e1ec948dcb7e89b9a046cccfc', 'request_id': 'req-10db2e20-5529-402f-8460-8e8782e59a44'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.554457 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f707372e1ec948dcb7e89b9a046cccfc', 'request_id': 'req-10db2e20-5529-402f-8460-8e8782e59a44'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.569648 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] The token cache is being invalidate because project f707372e1ec948dcb7e89b9a046cccfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:06:52.570774 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.571541 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.573687 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1476/2979] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:06:52 2022] DELETE /identity/v3/projects/f707372e1ec948dcb7e89b9a046cccfc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:06:52.577740 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:52.579594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:52.594347 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34860e5c-dc4c-4fd9-8f0a-ae9c29f56041 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:52.657014 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:52.657668 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:52.658017 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:52.658324 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] PATH_INFO: `/v3/users/320d32dac8e04f438d5be8f72e3c460a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:52.660021 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=320d32dac8e04f438d5be8f72e3c460a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:52.661969 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:52.668398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:06:52.726884 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320d32dac8e04f438d5be8f72e3c460a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.736577 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320d32dac8e04f438d5be8f72e3c460a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.737734 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320d32dac8e04f438d5be8f72e3c460a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.738547 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320d32dac8e04f438d5be8f72e3c460a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.752396 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c2a85d88-2e80-4578-a8a1-e9d3313f4918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320d32dac8e04f438d5be8f72e3c460a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:52.767755 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1504/2980] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:06:52 2022] DELETE /identity/v3/users/320d32dac8e04f438d5be8f72e3c460a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 18 07:06:52.954747 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:06:52.956421 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:06:52.990245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2e20-5529-402f-8460-8e8782e59a44 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:06:53.054702 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:06:53.055482 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:06:53.055894 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:06:53.056301 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] PATH_INFO: `/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:06:53.057725 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1728fc9adc45c3aa39654a0bb257e5)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:06:53.060097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:06:53.118126 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1728fc9adc45c3aa39654a0bb257e5', 'request_id': 'req-076c8b83-45f2-421b-9558-6cb84b5fb981'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:53.125843 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1728fc9adc45c3aa39654a0bb257e5', 'request_id': 'req-076c8b83-45f2-421b-9558-6cb84b5fb981'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:53.136200 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1728fc9adc45c3aa39654a0bb257e5', 'request_id': 'req-076c8b83-45f2-421b-9558-6cb84b5fb981'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:53.148168 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] The token cache is being invalidate because project 4f1728fc9adc45c3aa39654a0bb257e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:06:53.148555 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:53.149362 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:06:53.154860 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1477/2981] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:06:52 2022] DELETE /identity/v3/projects/4f1728fc9adc45c3aa39654a0bb257e5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:01.403848 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:01.404283 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:01.404416 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:01.424706 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] MFA Rules not processed for user `18bc18de709b424ca516b9082433f7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:07:01.425895 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:01.479964 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1505/2982] 10.222.0.20 () {58 vars in 967 bytes} [Sun Sep 18 07:07:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:07:01.492064 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:01.493735 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:01.536709 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:01.563507 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c8b83-45f2-421b-9558-6cb84b5fb981 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:01.568330 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd763c798dc824e92b5e51deb0e059ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:01.568919 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:01.569192 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:01.569955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:01.577801 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:01.579394 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:01.587858 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1478/2983] 10.222.0.20 () {60 vars in 1384 bytes} [Sun Sep 18 07:07:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 18 07:07:03.481196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:03.482757 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:03.548009 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c864981-c00a-4008-81cc-c1924ddbf72a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:03.552439 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:03.553056 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:03.553329 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:03.553594 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] PATH_INFO: `/v3/users/9842848655fe479bbea42fd0f2ca4ffa` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:03.554768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9842848655fe479bbea42fd0f2ca4ffa)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:03.556681 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:03.591579 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:03.606763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9842848655fe479bbea42fd0f2ca4ffa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.613629 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9842848655fe479bbea42fd0f2ca4ffa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.614237 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9842848655fe479bbea42fd0f2ca4ffa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.614868 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9842848655fe479bbea42fd0f2ca4ffa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.626758 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e65f48d8-27d8-47c9-a8b2-a78a5db9a0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9842848655fe479bbea42fd0f2ca4ffa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.639753 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1506/2984] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:07:03 2022] DELETE /identity/v3/users/9842848655fe479bbea42fd0f2ca4ffa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:03.809238 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:03.822173 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fe6b7-4400-4d97-990e-167288eb545b service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:03.878817 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:03.879496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:03.879806 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:03.880077 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] PATH_INFO: `/v3/projects/523fb59392af43bcb2be01e17406cef7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:03.881220 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=523fb59392af43bcb2be01e17406cef7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:03.883644 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:03.930351 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523fb59392af43bcb2be01e17406cef7', 'request_id': 'req-93fb5fc4-953a-439b-9594-8d1b8b6375f6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.936398 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523fb59392af43bcb2be01e17406cef7', 'request_id': 'req-93fb5fc4-953a-439b-9594-8d1b8b6375f6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.944054 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523fb59392af43bcb2be01e17406cef7', 'request_id': 'req-93fb5fc4-953a-439b-9594-8d1b8b6375f6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.954741 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] The token cache is being invalidate because project 523fb59392af43bcb2be01e17406cef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:07:03.955027 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.955638 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-93fb5fc4-953a-439b-9594-8d1b8b6375f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:03.957097 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1479/2985] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:07:03 2022] DELETE /identity/v3/projects/523fb59392af43bcb2be01e17406cef7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:03.961945 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:03.962228 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:03.962498 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:03.982037 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:07:03.983503 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:04.041565 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1507/2986] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:07:04.049062 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.049374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.049649 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.069984 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:07:04.071215 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:04.079042 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1480/2987] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:07:04.083196 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.093008 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5909f69-65de-4bfa-a9e5-488fb4688ca0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.097938 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.098543 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.098855 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.099122 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.099768 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 18 07:07:04.100067 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.101687 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.112947 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1508/2988] 10.222.0.20 () {58 vars in 1153 bytes} [Sun Sep 18 07:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 18 07:07:04.117915 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.127251 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca0c66-7e3f-4e49-92ab-8a8e092807cb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.132524 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.133090 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.133368 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.133636 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.134256 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.136203 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.162496 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1481/2989] 10.222.0.20 () {60 vars in 1153 bytes} [Sun Sep 18 07:07:04 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:07:04.166444 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.176177 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9385ce-1de6-4449-8f16-66b140b891a1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.233586 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.234090 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.234363 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.234659 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.235284 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.237193 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.259024 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1509/2990] 10.222.0.20 () {60 vars in 1144 bytes} [Sun Sep 18 07:07:04 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 18 07:07:04.263409 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.273777 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278095e-e1b9-4a46-933d-f0e407f2dc79 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.278411 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.279036 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.279343 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.279616 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.280255 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.282142 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.292581 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1482/2991] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:07:04.296379 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.306763 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f73b6-d232-4ac0-8096-70eae1232f23 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.311374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.311951 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.312225 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.312494 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.313166 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.314820 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12762e-6e22-41af-b63c-455bb3d9cdff admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.324674 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1510/2992] 10.222.0.20 () {58 vars in 1122 bytes} [Sun Sep 18 07:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 18 07:07:04.328775 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.338703 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484e7ea-924f-4dc0-90a4-3bb6f733f816 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.343327 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.343964 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.344245 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.344521 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] PATH_INFO: `/v3/projects/12b4f401c9494ce68f2451f7d76c3272/users/d928d85e6a604da199addb22c6f104e6/roles/d7498f66178947778670d4d051df8536` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.361007 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b4f401c9494ce68f2451f7d76c3272, user_id=d928d85e6a604da199addb22c6f104e6, role_id=d7498f66178947778670d4d051df8536)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.364688 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.368830 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:04.381610 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1483/2993] 10.222.0.20 () {60 vars in 1483 bytes} [Sun Sep 18 07:07:04 2022] PUT /identity/v3/projects/12b4f401c9494ce68f2451f7d76c3272/users/d928d85e6a604da199addb22c6f104e6/roles/d7498f66178947778670d4d051df8536 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:04.392398 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.392678 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.392943 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.418218 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:04.421546 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] MFA Rules not processed for user `d928d85e6a604da199addb22c6f104e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:07:04.423571 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:04.488221 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1511/2994] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:07:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:07:04.499450 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:04.501112 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:04.557534 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6759f-6248-44fd-baf3-707ddc5e093a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:04.561153 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db771368757c449887be80383cceedbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:04.561736 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:04.562011 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:04.562280 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:04.570374 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:04.571818 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775597fc-e026-4c89-b625-e18a0c971a46 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:04.580576 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1484/2995] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:07:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:07:14.485319 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=26 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:14.576528 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=28 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:14.713491 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:14.778236 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd667-da62-4207-8ec3-6f39eba28f96 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:14.782751 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:14.783374 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:14.783649 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:14.783918 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] PATH_INFO: `/v3/users/d928d85e6a604da199addb22c6f104e6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:14.785013 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d928d85e6a604da199addb22c6f104e6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:14.786972 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:14.832602 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd928d85e6a604da199addb22c6f104e6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:14.840393 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd928d85e6a604da199addb22c6f104e6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:14.841106 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd928d85e6a604da199addb22c6f104e6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:14.841847 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd928d85e6a604da199addb22c6f104e6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:14.853518 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd928d85e6a604da199addb22c6f104e6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:14.865295 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1512/2996] 10.222.0.20 () {58 vars in 1224 bytes} [Sun Sep 18 07:07:14 2022] DELETE /identity/v3/users/d928d85e6a604da199addb22c6f104e6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:14.870098 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:14.870433 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:14.870766 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:14.890955 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] MFA Rules not processed for user `949debb5f3604102a418027d122cd476`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 18 07:07:14.892163 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:14.950485 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1485/2997] 10.222.0.20 () {58 vars in 958 bytes} [Sun Sep 18 07:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 18 07:07:14.960782 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:14.962242 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:15.027288 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582e12-62d2-4eb6-9026-85b1d9d00f1b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:15.030887 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e742daf773049bf814cde4544b8a673', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:15.031479 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:15.031752 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:15.032020 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:15.040907 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:15.042318 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:15.051598 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1513/2998] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:07:15.313178 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:07:15.314618 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:07:15.326631 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121f14-62f5-4a41-a996-4e08a74a0cf4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:07:15.331397 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '949debb5f3604102a418027d122cd476', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '39ab4a771cc044ca930df75a3565efaa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:07:15.332024 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:07:15.332299 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:07:15.332570 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] PATH_INFO: `/v3/projects/12b4f401c9494ce68f2451f7d76c3272` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:07:15.333690 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b4f401c9494ce68f2451f7d76c3272)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:07:15.336117 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:07:15.381968 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b4f401c9494ce68f2451f7d76c3272', 'request_id': 'req-d176ddfa-3418-43fb-8175-8867bf78f597'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:15.388057 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b4f401c9494ce68f2451f7d76c3272', 'request_id': 'req-d176ddfa-3418-43fb-8175-8867bf78f597'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:15.395812 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b4f401c9494ce68f2451f7d76c3272', 'request_id': 'req-d176ddfa-3418-43fb-8175-8867bf78f597'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:15.406361 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] The token cache is being invalidate because project 12b4f401c9494ce68f2451f7d76c3272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 18 07:07:15.406684 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:15.407311 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d176ddfa-3418-43fb-8175-8867bf78f597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 18 07:07:15.409060 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1486/2999] 10.222.0.20 () {58 vars in 1233 bytes} [Sun Sep 18 07:07:15 2022] DELETE /identity/v3/projects/12b4f401c9494ce68f2451f7d76c3272 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 18 07:07:25.047450 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:07:25.403467 dv-cinder-855359-3-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 07:08:07.841138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 18 07:08:07.842540 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:08:07.930684 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4760b-abac-467d-b263-49c9703f3f07 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 18 07:08:07.987437 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc74075ebcdd4916aacec44e87e86064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '602a93d2c7f749dea6f26019f01dfa9c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 18 07:08:07.987965 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 18 07:08:07.988244 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 18 07:08:07.988514 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 18 07:08:07.990252 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 18 07:08:08.001519 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 18 07:08:08.003138 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e50239-a009-4891-a6ae-b622a389a144 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 18 07:08:08.013436 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1514/3000] 10.222.0.20 () {60 vars in 1365 bytes} [Sun Sep 18 07:08:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 18 07:08:18.009264 dv-cinder-855359-3-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=67 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}